Jul 24 20:13:22.036371 ip-10-0-136-110 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 20:13:22.036382 ip-10-0-136-110 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 20:13:22.036389 ip-10-0-136-110 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 20:13:22.036718 ip-10-0-136-110 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 20:13:32.174079 ip-10-0-136-110 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 20:13:32.174095 ip-10-0-136-110 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c8c5ff37496242b29c1dc71d280a1f81 --
Jul 24 20:15:32.340994 ip-10-0-136-110 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 20:15:32.710699 ip-10-0-136-110 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 20:15:32.710699 ip-10-0-136-110 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 20:15:32.710699 ip-10-0-136-110 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 20:15:32.710699 ip-10-0-136-110 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 20:15:32.710699 ip-10-0-136-110 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 20:15:32.711626 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.711540    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 20:15:32.715638 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715623    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.715638 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715639    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715643    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715646    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715649    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715652    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715655    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715658    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715661    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715664    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715667    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715669    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715672    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715676    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715679    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715681    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715684    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715687    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715690    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715692    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715695    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.715719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715698    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715701    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715703    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715719    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715722    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715725    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715729    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715731    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715734    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715737    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715740    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715743    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715745    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715748    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715750    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715753    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715756    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715758    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715761    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.716211 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715763    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715767    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715771    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715774    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715777    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715804    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715808    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715811    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715814    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715817    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715821    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715824    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715827    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715829    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715834    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715837    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715840    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715844    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715848    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.716665 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715851    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715853    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715856    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715859    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715861    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715864    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715867    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715870    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715873    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715876    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715878    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715881    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715884    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715887    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715890    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715893    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715896    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715899    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715901    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715904    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.717141 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715906    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715909    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715911    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715914    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715916    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715919    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715921    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715925    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715928    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715931    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715933    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715936    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715938    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715941    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715943    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715946    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715948    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715951    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715954    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.715956    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.717628 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716663    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716669    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716673    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716676    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716678    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716682    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716685    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716687    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716690    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716693    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716695    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716700    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716704    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716722    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716724    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716727    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716730    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716733    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716736    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716739    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.718120 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716742    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716744    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716747    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716750    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716752    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716755    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716758    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716760    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716763    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716765    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716768    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716771    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716773    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716776    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716778    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716782    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716786    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716789    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716793    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716796    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.718640 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716799    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716801    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716805    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716808    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716811    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716813    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716816    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716819    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716821    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716824    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716826    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716829    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716832    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716835    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716838    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716840    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716843    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716845    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716848    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.719143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716851    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716853    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716856    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716858    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716861    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716865    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716867    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716870    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716872    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716875    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716879    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716882    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716886    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716889    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716892    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716895    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716897    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716900    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716902    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.719639 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716905    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716908    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716911    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716913    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716916    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716921    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716923    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716926    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716929    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716931    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716934    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716937    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716939    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716942    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716944    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716947    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716950    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716952    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716955    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716957    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.720235 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.716960    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717346    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717354    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717357    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717360    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717363    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717367    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717370    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717372    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717375    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717378    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717380    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717383    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717385    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717388    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717391    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717394    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717398    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717400    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717403    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.720718 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717406    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717408    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717411    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717413    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717416    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717418    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717421    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717424    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717426    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717429    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717431    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717434    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717436    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717439    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717442    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717445    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717448    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717451    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717453    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.721247 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717456    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717460    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717464    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717467    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717470    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717472    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717475    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717477    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717480    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717483    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717486    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717490    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717492    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717495    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717499    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717503    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717507    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717511    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717514    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.721731 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717516    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717519    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717522    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717524    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717527    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717529    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717532    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717535    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717539    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717542    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717544    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717546    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717549    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717552    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717556    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717558    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717560    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717563    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717566    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717568    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.722215 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717571    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717573    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717576    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717579    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717582    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717585    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717588    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717590    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717593    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717595    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717598    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717600    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717603    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717605    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717608    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717610    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717613    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717615    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717618    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717620    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.722766 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717624    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.717626    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717689    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717695    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717701    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717719    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717724    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717727    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717731    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717736    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717739    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717742    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717745    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717749    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717752    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717756    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717759    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717761    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717764    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717767    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717771    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717773    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717776    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717779    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 20:15:32.723249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717783    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717786    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717789    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717792    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717795    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717798    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717801    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717804    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717808    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717811    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717814    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717818    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717820    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717823    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717826    2577 flags.go:64] FLAG: --enable-server="true"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717829    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717834    2577 flags.go:64] FLAG: --event-burst="100"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717837    2577 flags.go:64] FLAG: --event-qps="50"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717840    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717843    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717845    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717850    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717852    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717855    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717858    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 24 20:15:32.723828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717861    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717864    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717867    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717870    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717873    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717875    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717878    2577 flags.go:64] FLAG: --feature-gates=""
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717882    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717885    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717888    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717891    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717893    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717896    2577 flags.go:64] FLAG: --help="false"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717899    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-136-110.ec2.internal"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717902    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717907    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717909    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717913    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717916    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717919    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717922    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717925    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717928    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 20:15:32.724417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717932    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717935    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717938    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717941    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717944    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717947    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717950    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717953    2577 flags.go:64] FLAG: --lock-file=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717956    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717959    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717962    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717967    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717970    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717973    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717976    2577 flags.go:64] FLAG: --logging-format="text"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717978    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717982    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717985    2577 flags.go:64] FLAG: --manifest-url=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717988    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717992    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717995    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.717999    2577 flags.go:64] FLAG: --max-pods="110"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718002    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718005    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718009    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 20:15:32.725003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718012    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718015    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718018    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718021    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718029    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718032    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718035    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718039    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718042    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718047    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718050    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718053    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718056    2577 flags.go:64] FLAG: --port="10250"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718059    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718062    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-075e89180331480cb"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718066    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718068    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718071    2577 flags.go:64] FLAG: --register-node="true"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718074    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718077    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718080    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718083    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718085    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718089    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718092    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 20:15:32.725608 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718095    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718098    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718101    2577 flags.go:64] FLAG: --runonce="false"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718103    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718106    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718109    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718113    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718116    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718119    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718122    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718125    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718128    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718131    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718134    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718137    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718140    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718143    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718146    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718151    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718154    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718156    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718161    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718164    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718166    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718169    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 20:15:32.726258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718172    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718175    2577 flags.go:64] FLAG: --v="2"
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718179    2577 flags.go:64] FLAG: --version="false"
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718183    2577 flags.go:64] FLAG: --vmodule=""
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718186    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.718189    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718300    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718303    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718306    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718309    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718312    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718314    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718317    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718320    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718323    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718325    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718328    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718330    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718333    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718335    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718338    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.726892 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718341    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718343    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718346    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718349    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718351    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718354    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718356    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718360    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718362    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718365    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718367    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718370    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718372    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718375    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718377    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718380    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718383    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718386    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718388    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.727411 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718391    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718394    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718396    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718399    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718402    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718405    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718407    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718410    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718412    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718417    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718420    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718423    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718426    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718429    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718431    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718434    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718436    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718439    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718442    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718444    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.727886 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718448    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718451    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718455    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718458    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718461    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718464    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718466    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718469    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718471    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718474    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718477    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718479    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718482    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718485    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718487    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718490    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718492    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718495    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718497    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.728368 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718500    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718503    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718506    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718508    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718511    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718514    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718516    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718519    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718521    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718524    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718527    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718529    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718532    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718538    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718540    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718543    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718545    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718548    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718550    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.728862 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718553    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718555    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718558    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718561    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718564    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718567    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.718570    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.719205    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.725749    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.725764    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.725784    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.725789    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725843    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725848    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.729324 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725851    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725854    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725857    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725860    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725863    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725867    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725870    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725874    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725876    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725879    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725882    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725884    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725887    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725890    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725893    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725895    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725898    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725901    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725904    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.729678 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725906    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725909    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725911    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725915    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725918    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725921    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725923    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725926    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725930    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725933    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725935    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725938    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725940    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725943    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725945    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725948    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725952    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725955    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725958    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.730159 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725960    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725963    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725965    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725968    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725970    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725973    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725976    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725978    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725981    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725983    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725986    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725988    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725992    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725996    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.725998    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726001    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726004    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726006    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726009    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726011    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.730629 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726014    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726017    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726020    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726023    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726025    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726027    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726030    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726032    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726035    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726037    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726040    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726042    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726045    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726048    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726050    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726053    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726055    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726058    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726061    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726063    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.731133 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726066    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726068    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726071    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726073    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726076    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726078    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726081    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726083    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726086    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726088    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726091    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726093    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726096    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726099    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726102    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726104    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726107    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726110    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.731625 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726112    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.726117    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726230    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726234    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726237    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726240    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726242    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726245    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726247    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726250    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726252    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726255    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726257    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726262    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 20:15:32.732093 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726266    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726269    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726272    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726275    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726278    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726281    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726284    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726286    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726289    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726291    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726294    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726297    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726300    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726304    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726307    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726309    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726317    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726320    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726323    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726325    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 20:15:32.732450 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726328    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726330    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726333    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726335    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726338    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726340    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726343    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726346    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726348    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726351    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726353    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726356    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726358    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726361    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726363    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726366    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726369    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726371    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726374    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 20:15:32.733077 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726376    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726379    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726381    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726383    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726386    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726388    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726391    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726394    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726396    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726400    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726404    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726407    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726410    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726412    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726415    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726417    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726420    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726422    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726425    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 20:15:32.733537 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726427    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726430    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726432    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726434    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726437    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726439    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726442    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726444    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726447    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726449    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726452    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726454    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726457    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726459    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726461    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726464    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726466    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726469    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726471    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726477    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 20:15:32.734002 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726480    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726483    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726485    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726488    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726491    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726493    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726496    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726498    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:32.726500    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.726505    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.726616    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.728602    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.730159    2577 server.go:1019] "Starting client certificate rotation"
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.730261    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 20:15:32.734476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.731024    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 20:15:32.734840 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.732515    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 20:15:32.739172 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.739150    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 20:15:32.751697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.751675    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 24 20:15:32.753634 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.753621    2577 log.go:25] "Validated CRI v1 image API"
Jul 24 20:15:32.754410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.754398    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 20:15:32.757175 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.757157    2577 fs.go:135] Filesystem UUIDs: map[030bd9ed-d85a-464d-92f0-8dbc4f6208bf:/dev/nvme0n1p3 78486a59-9ca2-4c60-b837-a2557da28bf0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 24 20:15:32.757226 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.757196    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 20:15:32.761297 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.761193    2577 manager.go:216] Machine: {Timestamp:2026-07-24 20:15:32.760731941 +0000 UTC m=+0.302194735 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091686 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2051c64ea72db52e5074d76300f2c3 SystemUUID:ec2051c6-4ea7-2db5-2e50-74d76300f2c3 BootID:c8c5ff37-4962-42b2-9c1d-c71d280a1f81 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8b:f2:54:99:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8b:f2:54:99:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:68:5f:47:08:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 20:15:32.761297 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.761295    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 20:15:32.761407 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.761369    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 20:15:32.761747 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.761733    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 20:15:32.762612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.762576    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 20:15:32.762773 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.762616    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 20:15:32.763868 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.763857    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 20:15:32.763903 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.763871    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 20:15:32.763903 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.763879    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 20:15:32.764623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.764612    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 20:15:32.765677 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.765667    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 20:15:32.768438 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768426    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 20:15:32.768567 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768557    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 20:15:32.768602 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768587    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 20:15:32.768602 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768597    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 20:15:32.768655 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768616    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 20:15:32.768655 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768625    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 24 20:15:32.768655 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768633    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 20:15:32.768998 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.768981    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 20:15:32.768998 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.769000    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 20:15:32.772150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.772134    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 20:15:32.773861 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.773847    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 20:15:32.773925 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.773877    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 20:15:32.774452 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774442    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 20:15:32.774496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774456    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 20:15:32.774496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774463    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 20:15:32.774496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774468    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 20:15:32.774496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774476    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 20:15:32.774496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774481    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774538    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774546    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774552    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774558    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774566    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 20:15:32.774642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.774585    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 20:15:32.775178 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.775168    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 20:15:32.775178 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.775179    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 20:15:32.779402 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.779377    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 20:15:32.779402 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.779385    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 20:15:32.779526 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.779458    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 20:15:32.780771 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.780752    2577 server.go:1268] "Started kubelet"
Jul 24 20:15:32.780865 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.780822    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 20:15:32.781314 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.780878    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 20:15:32.781396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.781332    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 20:15:32.781626 ip-10-0-136-110 systemd[1]: Started Kubernetes Kubelet.
Jul 24 20:15:32.782042 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.782015    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qch96"
Jul 24 20:15:32.782988 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.782967    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 20:15:32.783571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.783554    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 20:15:32.788857 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.788841    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 20:15:32.789300 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.789287    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qch96"
Jul 24 20:15:32.789485 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.787732    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-110.ec2.internal.18c5530ab566c5a0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-110.ec2.internal,UID:ip-10-0-136-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-110.ec2.internal,},FirstTimestamp:2026-07-24 20:15:32.780459424 +0000 UTC m=+0.321922192,LastTimestamp:2026-07-24 20:15:32.780459424 +0000 UTC m=+0.321922192,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-110.ec2.internal,}"
Jul 24 20:15:32.790819 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.790801    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 20:15:32.790912 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.790820    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 20:15:32.790912 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.790866    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:32.791663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791523    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 20:15:32.791663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791541    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 20:15:32.791663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791524    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 20:15:32.791663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791655    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 20:15:32.791663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791663    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 20:15:32.791945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791695    2577 factory.go:55] Registering systemd factory
Jul 24 20:15:32.791945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791729    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 24 20:15:32.792010 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791960    2577 factory.go:153] Registering CRI-O factory
Jul 24 20:15:32.792010 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.791975    2577 factory.go:223] Registration of the crio container factory successfully
Jul 24 20:15:32.792204 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.792192    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 20:15:32.792251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.792219    2577 factory.go:103] Registering Raw factory
Jul 24 20:15:32.792251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.792233    2577 manager.go:1196] Started watching for new ooms in manager
Jul 24 20:15:32.792655 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.792640    2577 manager.go:349] Starting recovery of all containers
Jul 24 20:15:32.794962 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.794934    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 20:15:32.795102 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.795080    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 20:15:32.799981 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.799964    2577 manager.go:354] Recovery completed
Jul 24 20:15:32.806863 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.806849    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:32.808696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.808678    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:32.808774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.808729    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:32.808774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.808745    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:32.809277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.809263    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 20:15:32.809277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.809276    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 20:15:32.809371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.809277    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:32.809371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.809292    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 20:15:32.810034 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810018    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:32.810102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810049    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:32.810102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810061    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:32.810865 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.810775    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-110.ec2.internal.18c5530ab715a431  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-110.ec2.internal,UID:ip-10-0-136-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-110.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-110.ec2.internal,},FirstTimestamp:2026-07-24 20:15:32.808696881 +0000 UTC m=+0.350159667,LastTimestamp:2026-07-24 20:15:32.808696881 +0000 UTC m=+0.350159667,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-110.ec2.internal,}"
Jul 24 20:15:32.810973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810930    2577 policy_none.go:49] "None policy: Start"
Jul 24 20:15:32.810973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810954    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 20:15:32.810973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.810969    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 20:15:32.812081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.812069    2577 policy_none.go:47] "Start"
Jul 24 20:15:32.850292 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850266    2577 manager.go:337] "Starting Device Plugin manager"
Jul 24 20:15:32.850393 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.850308    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 20:15:32.850393 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850322    2577 server.go:87] "Starting device plugin registration server"
Jul 24 20:15:32.850618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850603    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 20:15:32.850677 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850620    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 20:15:32.850741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850728    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 20:15:32.850826 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850808    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 20:15:32.850826 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.850823    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 20:15:32.851385 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.851359    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 20:15:32.851458 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.851410    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:32.951247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.951216    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:32.952421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.952135    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:32.952421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.952166    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:32.952421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.952184    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:32.952421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.952212    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-110.ec2.internal"
Jul 24 20:15:32.956370 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.956345    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 20:15:32.957746 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.957730    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 20:15:32.957844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.957766    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 20:15:32.957844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.957786    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 20:15:32.957844 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.957825    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 20:15:32.959928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.959907    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:32.960215 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:32.960202    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-110.ec2.internal"
Jul 24 20:15:32.960257 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:32.960223    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-110.ec2.internal\": node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.006945 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.006889    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.058344 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.058320    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"]
Jul 24 20:15:33.058417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.058395    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:33.059259 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.059243    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:33.059318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.059274    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:33.059318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.059283    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:33.060503 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.060490    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:33.060640 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.060624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.060689 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.060678    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:33.061220 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061206    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:33.061290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061232    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:33.061290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061237    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:33.061290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061250    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:33.061290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061264    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:33.061420 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.061251    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:33.062416 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.062401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.062486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.062426    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 20:15:33.063081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.063064    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 20:15:33.063144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.063092    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 20:15:33.063144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.063108    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeHasSufficientPID"
Jul 24 20:15:33.089329 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.089311    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-110.ec2.internal\" not found" node="ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.093551 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.093534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.093628 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.093557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.093628 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.093581    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-110.ec2.internal\" not found" node="ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.093628 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.093592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8d8102a60824d951e2623901e81d027-config\") pod \"kube-apiserver-proxy-ip-10-0-136-110.ec2.internal\" (UID: \"e8d8102a60824d951e2623901e81d027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.107205 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.107182    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.194578 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8d8102a60824d951e2623901e81d027-config\") pod \"kube-apiserver-proxy-ip-10-0-136-110.ec2.internal\" (UID: \"e8d8102a60824d951e2623901e81d027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.194682 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.194682 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.194682 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8d8102a60824d951e2623901e81d027-config\") pod \"kube-apiserver-proxy-ip-10-0-136-110.ec2.internal\" (UID: \"e8d8102a60824d951e2623901e81d027\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.194828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.194828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.194782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8559fbd129c4e69f939af4a372324cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal\" (UID: \"f8559fbd129c4e69f939af4a372324cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.207719 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.207684    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.308590 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.308508    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.391762 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.391726    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.396839 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.396693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:33.409309 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.409290    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.509859 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.509827    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.610377 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.610319    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.675582 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.675561    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:33.710882 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.710857    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.729023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.729004    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 20:15:33.729126 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.729104    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 20:15:33.729183 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.729149    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 20:15:33.746289 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.746271    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:33.803648 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.803617    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 20:10:33 +0000 UTC" deadline="2028-04-23 00:35:00.991730694 +0000 UTC"
Jul 24 20:15:33.803648 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.803645    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15316h19m27.188088517s"
Jul 24 20:15:33.811778 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.811754    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.912456 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:33.912435    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-110.ec2.internal\" not found"
Jul 24 20:15:33.919277 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:33.919247    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d8102a60824d951e2623901e81d027.slice/crio-65848c4c1b38065328a48f7728aec536de084bbd7eb8947a24251e344e5af398 WatchSource:0}: Error finding container 65848c4c1b38065328a48f7728aec536de084bbd7eb8947a24251e344e5af398: Status 404 returned error can't find the container with id 65848c4c1b38065328a48f7728aec536de084bbd7eb8947a24251e344e5af398
Jul 24 20:15:33.919516 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:33.919496    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8559fbd129c4e69f939af4a372324cf.slice/crio-21d5b11af9a87d3b81aa215e16cc84d310b334f3eab36ccb39a180673ca49d1b WatchSource:0}: Error finding container 21d5b11af9a87d3b81aa215e16cc84d310b334f3eab36ccb39a180673ca49d1b: Status 404 returned error can't find the container with id 21d5b11af9a87d3b81aa215e16cc84d310b334f3eab36ccb39a180673ca49d1b
Jul 24 20:15:33.924975 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.924961    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:15:33.943208 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.943191    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:33.961402 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.961362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal" event={"ID":"e8d8102a60824d951e2623901e81d027","Type":"ContainerStarted","Data":"65848c4c1b38065328a48f7728aec536de084bbd7eb8947a24251e344e5af398"}
Jul 24 20:15:33.962473 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.962450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal" event={"ID":"f8559fbd129c4e69f939af4a372324cf","Type":"ContainerStarted","Data":"21d5b11af9a87d3b81aa215e16cc84d310b334f3eab36ccb39a180673ca49d1b"}
Jul 24 20:15:33.991626 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:33.991605    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:34.008932 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:34.008909    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 20:15:34.009908 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:34.009896    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal"
Jul 24 20:15:34.033070 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:34.033052    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 20:15:35.958014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.957985    2577 apiserver.go:52] "Watching apiserver"
Jul 24 20:15:35.971223 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.971195    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 20:15:35.971615 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.971570    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal","openshift-multus/multus-additional-cni-plugins-9px48","openshift-multus/multus-s9b8j","openshift-ovn-kubernetes/ovnkube-node-hxdhf","kube-system/global-pull-secret-syncer-nxgsn","kube-system/konnectivity-agent-n5gpr","openshift-image-registry/node-ca-9t6cs","openshift-multus/network-metrics-daemon-2tlc7","openshift-network-diagnostics/network-check-target-m6v27","openshift-network-operator/iptables-alerter-p9rgq","kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4","openshift-cluster-node-tuning-operator/tuned-t7v97","openshift-dns/node-resolver-4rcfk"]
Jul 24 20:15:35.973219 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.973196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:35.974897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.974877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.975770 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.975745    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 20:15:35.975889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.975856    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.975954 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.975910    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kxddx\""
Jul 24 20:15:35.975954 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.975949    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.976277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.976258    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.977670 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.977497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.978315 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.978287    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.978534 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.978516    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.978649 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.978547    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zw9f6\""
Jul 24 20:15:35.978701 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.978659    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 20:15:35.979189 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.979174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cc9lj\""
Jul 24 20:15:35.979674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.979654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:35.979763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.979588    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 20:15:35.979763 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:35.979742    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:35.981157 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.980861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.981157 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.980871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:35.981157 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.980936    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 20:15:35.982064 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.982047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:35.982153 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:35.982120    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:35.983256 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:35.983671 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983501    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 20:15:35.983671 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 20:15:35.983799 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mv49b\""
Jul 24 20:15:35.983799 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 20:15:35.983874 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983848    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.983979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983958    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 20:15:35.984068 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.984015    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.984068 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.984048    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.984149 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.983961    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 20:15:35.984212 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.984189    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 20:15:35.984278 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.984227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qr2hb\""
Jul 24 20:15:35.984438 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.984420    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:35.985566 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.985551    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.985895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.985877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:35.985895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.985891    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.986030 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.985898    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hvdnn\""
Jul 24 20:15:35.987262 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.987245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:35.987368 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:35.987303    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:35.987430 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.987406    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 20:15:35.987696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.987407    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 20:15:35.987696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.987516    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2x9wh\""
Jul 24 20:15:35.988730 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.988707    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 20:15:35.988846 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.988827    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:35.989151 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.989130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-462vt\""
Jul 24 20:15:35.990817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.990788    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.990817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.990802    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.992506 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-lib-modules\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.992623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bccd7a2-f090-4038-ab98-1d605c39c415-konnectivity-ca\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:35.992623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.992623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-socket-dir-parent\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-slash\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-var-lib-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-conf\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-host\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-multus\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.992774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-k8s-cni-cncf-io\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-var-lib-kubelet\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysconfig\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-run\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-tmp\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-bin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-conf-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.992985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-daemon-config\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-config\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-env-overrides\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-script-lib\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6f0789c-a141-42b5-a35f-5d61f8e2188e-host\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-netns\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-bin\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993462    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5kvjf\""
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6180745c-de33-463c-834f-fb1c7abde2de-ovn-node-metrics-cert\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cni-binary-copy\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-tuned\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvqtc\" (UniqueName: \"kubernetes.io/projected/95f772b9-b49a-4d4e-b076-c9af427f5a75-kube-api-access-xvqtc\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-os-release\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-kubelet\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-multus-certs\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-kubelet-config\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993758    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqt6p\" (UniqueName: \"kubernetes.io/projected/4446fcea-0da3-4c00-aeaf-18a9a01770ad-kube-api-access-hqt6p\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993806    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-systemd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv5nk\" (UniqueName: \"kubernetes.io/projected/6180745c-de33-463c-834f-fb1c7abde2de-kube-api-access-zv5nk\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993895    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 20:15:35.993969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.993900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6f0789c-a141-42b5-a35f-5d61f8e2188e-serviceca\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-kubelet\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-systemd-units\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-node-log\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-log-socket\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bccd7a2-f090-4038-ab98-1d605c39c415-agent-certs\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-netd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-kubernetes\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-sys\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.994512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-etc-kubernetes\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.994968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-etc-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-ovn\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.994968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdp5n\" (UniqueName: \"kubernetes.io/projected/a6f0789c-a141-42b5-a35f-5d61f8e2188e-kube-api-access-wdp5n\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:35.994968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.994938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-systemd\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.995439 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-system-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.995523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-netns\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.995523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-hostroot\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.995523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-modprobe-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:35.995523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-dbus\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:35.995712 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:35.995712 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.995557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cnibin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:35.996858 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:35.996839    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 20:15:36.018954 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.018930    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p9b2w"
Jul 24 20:15:36.028066 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.028048    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p9b2w"
Jul 24 20:15:36.080577 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.080550    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 20:15:36.095930 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.095904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-sys\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.095941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-etc-kubernetes\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.095966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-etc-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.095986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-ovn\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad07c18d-61bb-43c7-9e8c-08506c9968ad-tmp-dir\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-sys\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.096063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t6kt\" (UniqueName: \"kubernetes.io/projected/48e31ee6-5de8-487f-b301-518d5d7b09c3-kube-api-access-2t6kt\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-etc-kubernetes\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-ovn\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-etc-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdp5n\" (UniqueName: \"kubernetes.io/projected/a6f0789c-a141-42b5-a35f-5d61f8e2188e-kube-api-access-wdp5n\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-systemd\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-system-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-netns\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-systemd\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-hostroot\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-system-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad07c18d-61bb-43c7-9e8c-08506c9968ad-hosts-file\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-netns\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-system-cni-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-hostroot\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.096369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-modprobe-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-dbus\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shrwq\" (UniqueName: \"kubernetes.io/projected/c249c9d6-0606-4b78-9ce7-281e837c2bee-kube-api-access-shrwq\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-registration-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-modprobe-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cnibin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-dbus\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk6mq\" (UniqueName: \"kubernetes.io/projected/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-kube-api-access-bk6mq\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cnibin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-socket-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-lib-modules\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bccd7a2-f090-4038-ab98-1d605c39c415-konnectivity-ca\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-lib-modules\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.096980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-d\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-socket-dir-parent\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-socket-dir-parent\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-slash\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-slash\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-var-lib-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-var-lib-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-host-slash\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bccd7a2-f090-4038-ab98-1d605c39c415-konnectivity-ca\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-etc-selinux\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-conf\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-host\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-multus\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-multus\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-host\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysctl-conf\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.097696 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-cnibin\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-k8s-cni-cncf-io\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-var-lib-kubelet\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-sys-fs\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysconfig\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-run\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-tmp\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-bin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-conf-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-daemon-config\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.097991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-config\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-env-overrides\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbhrd\" (UniqueName: \"kubernetes.io/projected/ad07c18d-61bb-43c7-9e8c-08506c9968ad-kube-api-access-cbhrd\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-run\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-script-lib\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6f0789c-a141-42b5-a35f-5d61f8e2188e-host\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-netns\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52tx\" (UniqueName: \"kubernetes.io/projected/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kube-api-access-b52tx\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-bin\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6180745c-de33-463c-834f-fb1c7abde2de-ovn-node-metrics-cert\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cni-binary-copy\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-tuned\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvqtc\" (UniqueName: \"kubernetes.io/projected/95f772b9-b49a-4d4e-b076-c9af427f5a75-kube-api-access-xvqtc\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-os-release\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-kubelet\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-os-release\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-multus-certs\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-iptables-alerter-script\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.098972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-kubelet-config\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-device-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqt6p\" (UniqueName: \"kubernetes.io/projected/4446fcea-0da3-4c00-aeaf-18a9a01770ad-kube-api-access-hqt6p\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-systemd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv5nk\" (UniqueName: \"kubernetes.io/projected/6180745c-de33-463c-834f-fb1c7abde2de-kube-api-access-zv5nk\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-script-lib\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6f0789c-a141-42b5-a35f-5d61f8e2188e-serviceca\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-kubelet\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-systemd-units\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-node-log\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-log-socket\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bccd7a2-f090-4038-ab98-1d605c39c415-agent-certs\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-netd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.099757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-kubernetes\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-kubernetes\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098874    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-k8s-cni-cncf-io\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.098999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-var-lib-kubelet\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-sysconfig\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6f0789c-a141-42b5-a35f-5d61f8e2188e-host\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-run-netns\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-bin\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-cni-bin\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-cni-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.099785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-conf-dir\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.100306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-multus-daemon-config\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.100328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4446fcea-0da3-4c00-aeaf-18a9a01770ad-cni-binary-copy\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.100367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-openvswitch\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.100530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.100413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.101280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.100881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-ovnkube-config\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.101280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-os-release\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.101280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6180745c-de33-463c-834f-fb1c7abde2de-env-overrides\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.101280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-var-lib-kubelet\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.101280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4446fcea-0da3-4c00-aeaf-18a9a01770ad-host-run-multus-certs\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.101519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/502e4008-5e5a-4bee-8936-c5f799313c4f-kubelet-config\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.101661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-run-systemd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.101812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-systemd-units\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.101812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.101688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-node-log\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.102358 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.101966    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:36.102358 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.102038    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:15:36.602016445 +0000 UTC m=+2.955593585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:36.102548 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.102527    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-cni-netd\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.102631 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.102571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6f0789c-a141-42b5-a35f-5d61f8e2188e-serviceca\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.102631 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.102614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-log-socket\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.102712 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.102664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6180745c-de33-463c-834f-fb1c7abde2de-host-kubelet\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.102763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.102714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-etc-tuned\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.103294 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.103275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f772b9-b49a-4d4e-b076-c9af427f5a75-tmp\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.104614 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.104553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6180745c-de33-463c-834f-fb1c7abde2de-ovn-node-metrics-cert\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.104985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.104967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdp5n\" (UniqueName: \"kubernetes.io/projected/a6f0789c-a141-42b5-a35f-5d61f8e2188e-kube-api-access-wdp5n\") pod \"node-ca-9t6cs\" (UID: \"a6f0789c-a141-42b5-a35f-5d61f8e2188e\") " pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.105055 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.105020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bccd7a2-f090-4038-ab98-1d605c39c415-agent-certs\") pod \"konnectivity-agent-n5gpr\" (UID: \"6bccd7a2-f090-4038-ab98-1d605c39c415\") " pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:36.109020 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.108997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqt6p\" (UniqueName: \"kubernetes.io/projected/4446fcea-0da3-4c00-aeaf-18a9a01770ad-kube-api-access-hqt6p\") pod \"multus-s9b8j\" (UID: \"4446fcea-0da3-4c00-aeaf-18a9a01770ad\") " pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.109422 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.109398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv5nk\" (UniqueName: \"kubernetes.io/projected/6180745c-de33-463c-834f-fb1c7abde2de-kube-api-access-zv5nk\") pod \"ovnkube-node-hxdhf\" (UID: \"6180745c-de33-463c-834f-fb1c7abde2de\") " pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.109846 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.109827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvqtc\" (UniqueName: \"kubernetes.io/projected/95f772b9-b49a-4d4e-b076-c9af427f5a75-kube-api-access-xvqtc\") pod \"tuned-t7v97\" (UID: \"95f772b9-b49a-4d4e-b076-c9af427f5a75\") " pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.199449 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.199449 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad07c18d-61bb-43c7-9e8c-08506c9968ad-tmp-dir\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t6kt\" (UniqueName: \"kubernetes.io/projected/48e31ee6-5de8-487f-b301-518d5d7b09c3-kube-api-access-2t6kt\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad07c18d-61bb-43c7-9e8c-08506c9968ad-hosts-file\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-system-cni-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shrwq\" (UniqueName: \"kubernetes.io/projected/c249c9d6-0606-4b78-9ce7-281e837c2bee-kube-api-access-shrwq\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-registration-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk6mq\" (UniqueName: \"kubernetes.io/projected/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-kube-api-access-bk6mq\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.199711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-socket-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-host-slash\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-etc-selinux\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-cnibin\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad07c18d-61bb-43c7-9e8c-08506c9968ad-tmp-dir\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-registration-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-sys-fs\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad07c18d-61bb-43c7-9e8c-08506c9968ad-hosts-file\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-system-cni-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbhrd\" (UniqueName: \"kubernetes.io/projected/ad07c18d-61bb-43c7-9e8c-08506c9968ad-kube-api-access-cbhrd\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52tx\" (UniqueName: \"kubernetes.io/projected/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kube-api-access-b52tx\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-os-release\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-etc-selinux\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.199982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-iptables-alerter-script\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.200091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-device-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-sys-fs\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-device-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-host-slash\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.200220    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-cnibin\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.200264    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:15:36.700252148 +0000 UTC m=+3.053829266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9dd76d27-beeb-4fd2-bab0-9d2411990f76-socket-dir\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-iptables-alerter-script\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e31ee6-5de8-487f-b301-518d5d7b09c3-os-release\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.200885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.201439 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.200970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e31ee6-5de8-487f-b301-518d5d7b09c3-cni-binary-copy\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.207026 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.207004    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:36.207026 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.207029    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:36.207196 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.207044    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:36.207196 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.207112    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:15:36.707095302 +0000 UTC m=+3.060672450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:36.208524 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.208466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t6kt\" (UniqueName: \"kubernetes.io/projected/48e31ee6-5de8-487f-b301-518d5d7b09c3-kube-api-access-2t6kt\") pod \"multus-additional-cni-plugins-9px48\" (UID: \"48e31ee6-5de8-487f-b301-518d5d7b09c3\") " pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.210907 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.210885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk6mq\" (UniqueName: \"kubernetes.io/projected/e9654e1a-3b99-487d-9d8f-e1bba3e986e1-kube-api-access-bk6mq\") pod \"iptables-alerter-p9rgq\" (UID: \"e9654e1a-3b99-487d-9d8f-e1bba3e986e1\") " pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.211029 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.211007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shrwq\" (UniqueName: \"kubernetes.io/projected/c249c9d6-0606-4b78-9ce7-281e837c2bee-kube-api-access-shrwq\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.212262 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.212239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52tx\" (UniqueName: \"kubernetes.io/projected/9dd76d27-beeb-4fd2-bab0-9d2411990f76-kube-api-access-b52tx\") pod \"aws-ebs-csi-driver-node-kctv4\" (UID: \"9dd76d27-beeb-4fd2-bab0-9d2411990f76\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.212348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.212287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbhrd\" (UniqueName: \"kubernetes.io/projected/ad07c18d-61bb-43c7-9e8c-08506c9968ad-kube-api-access-cbhrd\") pod \"node-resolver-4rcfk\" (UID: \"ad07c18d-61bb-43c7-9e8c-08506c9968ad\") " pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.285043 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.285012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9t6cs"
Jul 24 20:15:36.295252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.295227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t7v97"
Jul 24 20:15:36.316766 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.316743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p9rgq"
Jul 24 20:15:36.322473 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.322445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9b8j"
Jul 24 20:15:36.329557 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.329540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:15:36.334328 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.334306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:36.340985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.340967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rcfk"
Jul 24 20:15:36.347389 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.347359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9px48"
Jul 24 20:15:36.348474 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.348456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4"
Jul 24 20:15:36.432277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.432248    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:36.602884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.602812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:36.603024 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.602959    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:36.603072 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.603037    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:15:37.603013368 +0000 UTC m=+3.956590541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:36.703397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.703357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:36.703560 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.703521    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:36.703659 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.703607    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:15:37.703571975 +0000 UTC m=+4.057149094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:36.804157 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:36.804120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:36.804320 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.804262    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:36.804320 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.804279    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:36.804320 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.804287    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:36.804450 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:36.804344    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:15:37.804325132 +0000 UTC m=+4.157902250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:36.901307 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.901268    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4446fcea_0da3_4c00_aeaf_18a9a01770ad.slice/crio-af0bed220d00d53767639ae3ad18e128582d592877f92a7f37cc4b17208f5786 WatchSource:0}: Error finding container af0bed220d00d53767639ae3ad18e128582d592877f92a7f37cc4b17208f5786: Status 404 returned error can't find the container with id af0bed220d00d53767639ae3ad18e128582d592877f92a7f37cc4b17208f5786
Jul 24 20:15:36.902195 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.902080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bccd7a2_f090_4038_ab98_1d605c39c415.slice/crio-5eb02319d78cbd3c1ab129e89902d1cc7d49e0dea4da507910ad9b213528a94a WatchSource:0}: Error finding container 5eb02319d78cbd3c1ab129e89902d1cc7d49e0dea4da507910ad9b213528a94a: Status 404 returned error can't find the container with id 5eb02319d78cbd3c1ab129e89902d1cc7d49e0dea4da507910ad9b213528a94a
Jul 24 20:15:36.905171 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.905137    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e31ee6_5de8_487f_b301_518d5d7b09c3.slice/crio-e9a36d0475313883a8d8e459a173d46db8a70aab6947e54185ea8812e59dc6b4 WatchSource:0}: Error finding container e9a36d0475313883a8d8e459a173d46db8a70aab6947e54185ea8812e59dc6b4: Status 404 returned error can't find the container with id e9a36d0475313883a8d8e459a173d46db8a70aab6947e54185ea8812e59dc6b4
Jul 24 20:15:36.907549 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.907528    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f0789c_a141_42b5_a35f_5d61f8e2188e.slice/crio-8755567b7c826984cf7ca3ada4e70b26bfe2beed5efa3ee1e1b5635060a18a9c WatchSource:0}: Error finding container 8755567b7c826984cf7ca3ada4e70b26bfe2beed5efa3ee1e1b5635060a18a9c: Status 404 returned error can't find the container with id 8755567b7c826984cf7ca3ada4e70b26bfe2beed5efa3ee1e1b5635060a18a9c
Jul 24 20:15:36.908363 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.908330    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9654e1a_3b99_487d_9d8f_e1bba3e986e1.slice/crio-d3a77c1a768dbf91b894b8fdede2686914f0d2d517fbf3524f3b4cae03b01b03 WatchSource:0}: Error finding container d3a77c1a768dbf91b894b8fdede2686914f0d2d517fbf3524f3b4cae03b01b03: Status 404 returned error can't find the container with id d3a77c1a768dbf91b894b8fdede2686914f0d2d517fbf3524f3b4cae03b01b03
Jul 24 20:15:36.909167 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.909148    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad07c18d_61bb_43c7_9e8c_08506c9968ad.slice/crio-860f47b753a1767d22f1070aea18e094e220470fd6ee98ef174a38ad7dffac5c WatchSource:0}: Error finding container 860f47b753a1767d22f1070aea18e094e220470fd6ee98ef174a38ad7dffac5c: Status 404 returned error can't find the container with id 860f47b753a1767d22f1070aea18e094e220470fd6ee98ef174a38ad7dffac5c
Jul 24 20:15:36.912193 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.910473    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6180745c_de33_463c_834f_fb1c7abde2de.slice/crio-7cfc5f55fe3759a838c4599e52165fc6308d63985591296cbb78d9500208463f WatchSource:0}: Error finding container 7cfc5f55fe3759a838c4599e52165fc6308d63985591296cbb78d9500208463f: Status 404 returned error can't find the container with id 7cfc5f55fe3759a838c4599e52165fc6308d63985591296cbb78d9500208463f
Jul 24 20:15:36.913088 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:15:36.912711    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f772b9_b49a_4d4e_b076_c9af427f5a75.slice/crio-37b535e8add31ec949d5e3f2d7040d23f72e9087c4260005534cd7541ff62f51 WatchSource:0}: Error finding container 37b535e8add31ec949d5e3f2d7040d23f72e9087c4260005534cd7541ff62f51: Status 404 returned error can't find the container with id 37b535e8add31ec949d5e3f2d7040d23f72e9087c4260005534cd7541ff62f51
Jul 24 20:15:37.031010 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.030980    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 20:10:36 +0000 UTC" deadline="2028-02-28 07:32:31.74357132 +0000 UTC"
Jul 24 20:15:37.031010 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.031006    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14003h16m54.712568091s"
Jul 24 20:15:37.146396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.146156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:37.146396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.146225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:37.146571 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.146413    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:37.146571 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.146502    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:37.154367 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.154343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"7cfc5f55fe3759a838c4599e52165fc6308d63985591296cbb78d9500208463f"}
Jul 24 20:15:37.155377 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.155356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rcfk" event={"ID":"ad07c18d-61bb-43c7-9e8c-08506c9968ad","Type":"ContainerStarted","Data":"860f47b753a1767d22f1070aea18e094e220470fd6ee98ef174a38ad7dffac5c"}
Jul 24 20:15:37.156318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.156297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9b8j" event={"ID":"4446fcea-0da3-4c00-aeaf-18a9a01770ad","Type":"ContainerStarted","Data":"af0bed220d00d53767639ae3ad18e128582d592877f92a7f37cc4b17208f5786"}
Jul 24 20:15:37.157314 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.157294    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4" event={"ID":"9dd76d27-beeb-4fd2-bab0-9d2411990f76","Type":"ContainerStarted","Data":"3943b02f3d69f101ecf8f44f738781c587f6f118404cb1ab8ab325e8f784aae8"}
Jul 24 20:15:37.158318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.158298    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p9rgq" event={"ID":"e9654e1a-3b99-487d-9d8f-e1bba3e986e1","Type":"ContainerStarted","Data":"d3a77c1a768dbf91b894b8fdede2686914f0d2d517fbf3524f3b4cae03b01b03"}
Jul 24 20:15:37.159243 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.159225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9t6cs" event={"ID":"a6f0789c-a141-42b5-a35f-5d61f8e2188e","Type":"ContainerStarted","Data":"8755567b7c826984cf7ca3ada4e70b26bfe2beed5efa3ee1e1b5635060a18a9c"}
Jul 24 20:15:37.160159 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.160141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerStarted","Data":"e9a36d0475313883a8d8e459a173d46db8a70aab6947e54185ea8812e59dc6b4"}
Jul 24 20:15:37.161075 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.161059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n5gpr" event={"ID":"6bccd7a2-f090-4038-ab98-1d605c39c415","Type":"ContainerStarted","Data":"5eb02319d78cbd3c1ab129e89902d1cc7d49e0dea4da507910ad9b213528a94a"}
Jul 24 20:15:37.162736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.162718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal" event={"ID":"e8d8102a60824d951e2623901e81d027","Type":"ContainerStarted","Data":"ba9a56a7686ccd9c4f97cca82ee28948aa150745410d755f3516632a6f907c2f"}
Jul 24 20:15:37.164287 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.164264    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8559fbd129c4e69f939af4a372324cf" containerID="fd9534670db983adeaefcd5526ecfffe8392e0a9ac876a511e017a8ded561b3f" exitCode=0
Jul 24 20:15:37.164349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.164327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal" event={"ID":"f8559fbd129c4e69f939af4a372324cf","Type":"ContainerDied","Data":"fd9534670db983adeaefcd5526ecfffe8392e0a9ac876a511e017a8ded561b3f"}
Jul 24 20:15:37.165345 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.165326    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7v97" event={"ID":"95f772b9-b49a-4d4e-b076-c9af427f5a75","Type":"ContainerStarted","Data":"37b535e8add31ec949d5e3f2d7040d23f72e9087c4260005534cd7541ff62f51"}
Jul 24 20:15:37.181666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.181589    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-110.ec2.internal" podStartSLOduration=2.181571414 podStartE2EDuration="2.181571414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:37.18144567 +0000 UTC m=+3.535022812" watchObservedRunningTime="2026-07-24 20:15:37.181571414 +0000 UTC m=+3.535148556"
Jul 24 20:15:37.610788 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.610416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:37.610788 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.610577    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:37.610788 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.610664    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:15:39.61064539 +0000 UTC m=+5.964222511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:37.711803 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.711751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:37.712016 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.711997    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:37.712102 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.712060    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:15:39.712039451 +0000 UTC m=+6.065616575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:37.725881 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.725854    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 20:15:37.813026 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:37.812940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:37.813190 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.813117    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:37.813190 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.813136    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:37.813190 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.813148    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:37.813351 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:37.813210    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:15:39.813191929 +0000 UTC m=+6.166769070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:38.031933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:38.031710    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 20:10:36 +0000 UTC" deadline="2028-04-14 17:53:22.381712228 +0000 UTC"
Jul 24 20:15:38.031933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:38.031774    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15117h37m44.349942218s"
Jul 24 20:15:38.149454 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:38.148879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:38.149454 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:38.149004    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:38.181827 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:38.181753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal" event={"ID":"f8559fbd129c4e69f939af4a372324cf","Type":"ContainerStarted","Data":"33bd978b75d7a93cc767e5808be83cf0b754a2409fd8bd07ca5bbd07489f33c5"}
Jul 24 20:15:38.200203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:38.199770    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-110.ec2.internal" podStartSLOduration=3.199749632 podStartE2EDuration="3.199749632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:38.199336238 +0000 UTC m=+4.552913379" watchObservedRunningTime="2026-07-24 20:15:38.199749632 +0000 UTC m=+4.553326776"
Jul 24 20:15:39.146109 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:39.146077    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:39.146528 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.146209    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:39.146528 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:39.146273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:39.146528 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.146358    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:39.628830 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:39.628749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:39.628990 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.628895    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:39.628990 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.628965    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:15:43.628945705 +0000 UTC m=+9.982522838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:39.729702 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:39.729663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:39.729878 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.729812    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:39.729939 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.729892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:15:43.729870941 +0000 UTC m=+10.083448058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:39.830074 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:39.830036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:39.830250 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.830194    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:39.830250 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.830213    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:39.830250 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.830225    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:39.830383 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:39.830292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:15:43.830273599 +0000 UTC m=+10.183850746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:40.147621 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:40.146056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:40.147621 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:40.146187    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:41.146694 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:41.146652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:41.146867 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:41.146775    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:41.147114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:41.146652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:41.147248 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:41.147221    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:42.146915 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:42.146884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:42.147358 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:42.147002    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:43.146674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:43.146645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:43.146674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:43.146675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:43.146921 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.146766    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:43.147169 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.147143    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:43.665106 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:43.664516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:43.665106 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.664706    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:43.665106 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.664770    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:15:51.664749496 +0000 UTC m=+18.018326638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:43.765269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:43.765230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:43.765446 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.765364    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:43.765446 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.765442    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:15:51.76542313 +0000 UTC m=+18.119000262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:43.866672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:43.866609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:43.866851 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.866761    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:43.866851 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.866787    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:43.866851 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.866801    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:43.866957 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:43.866863    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:15:51.866842674 +0000 UTC m=+18.220419798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:44.147166 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:44.147089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:44.147589 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:44.147187    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:45.146498 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:45.146454    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:45.146674 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:45.146623    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:45.147040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:45.147017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:45.147183 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:45.147115    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:46.155034 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:46.155001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:46.155494 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:46.155124    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:47.146198 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:47.146164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:47.146332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:47.146202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:47.146332 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:47.146274    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:47.146415 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:47.146374    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:48.146274 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:48.146238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:48.146743 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:48.146356    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:49.146562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:49.146530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:49.147030 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:49.146540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:49.147030 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:49.146672    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:49.147030 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:49.146715    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:50.146419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:50.146386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:50.146632 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:50.146521    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:51.146650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:51.146609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:51.147147 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.146743    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:51.147147 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:51.146800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:51.147147 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.146899    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:51.723614 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:51.723566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:51.723798 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.723711    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:51.723798 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.723785    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:16:07.723765345 +0000 UTC m=+34.077342472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:15:51.823944 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:51.823916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:51.824118 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.824036    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:51.824118 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.824100    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:16:07.824086245 +0000 UTC m=+34.177663363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:15:51.924966 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:51.924935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:51.925149 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.925095    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:15:51.925149 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.925120    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:15:51.925149 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.925133    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:51.925270 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:51.925194    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:16:07.92517449 +0000 UTC m=+34.278751613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:15:52.146210 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:52.146118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:52.146362 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:52.146246    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:53.146557 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:53.146525    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:53.146990 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:53.146540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:53.146990 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:53.146658    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:53.146990 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:53.146755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:54.147265 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:54.147231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:54.147661 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:54.147364    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:55.146814 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:55.146781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:55.146814 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:55.146803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:55.147048 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:55.146894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:55.147117 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:55.147047    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:56.146854 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.146686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:56.147350 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:56.146945    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:56.222929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.222897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9t6cs" event={"ID":"a6f0789c-a141-42b5-a35f-5d61f8e2188e","Type":"ContainerStarted","Data":"e7598a298bad12e47ac38602b739d586b74f0dade9b3cbedd02573bf28024904"}
Jul 24 20:15:56.226261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.226236    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="4553bbf6bff9793baf846a9af362b604e6b996c1a6917754a8e8e8c3b95b2b04" exitCode=0
Jul 24 20:15:56.226380 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.226308    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"4553bbf6bff9793baf846a9af362b604e6b996c1a6917754a8e8e8c3b95b2b04"}
Jul 24 20:15:56.228228 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.227937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n5gpr" event={"ID":"6bccd7a2-f090-4038-ab98-1d605c39c415","Type":"ContainerStarted","Data":"eea6f3d2b1373b323eb220211eddb27f39a97e65ae11d93110f264b4da3af9b4"}
Jul 24 20:15:56.229392 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.229362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t7v97" event={"ID":"95f772b9-b49a-4d4e-b076-c9af427f5a75","Type":"ContainerStarted","Data":"4933bfbc7670585768a8b51396063d3bda3e7620341799e0fe2becfa169f3d87"}
Jul 24 20:15:56.231142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.231123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"a02b1623c316d7e8f6109c9464c38106548c013589d4ba1b9abbf6720ff1dda0"}
Jul 24 20:15:56.231221 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.231188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"08ad027e17be73be35847aae255719a531a06beead136bdedf69619272007d4e"}
Jul 24 20:15:56.232422 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.232404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rcfk" event={"ID":"ad07c18d-61bb-43c7-9e8c-08506c9968ad","Type":"ContainerStarted","Data":"8c0f3bc9c93bbf9043445b7333c697816e5fdd273bd134020266f8e0ae09f4b7"}
Jul 24 20:15:56.233752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.233730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9b8j" event={"ID":"4446fcea-0da3-4c00-aeaf-18a9a01770ad","Type":"ContainerStarted","Data":"7e46cf15b43d86569600969e6e45a0b6f70e8ab9741076bcf19dae18265979f5"}
Jul 24 20:15:56.234979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.234960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4" event={"ID":"9dd76d27-beeb-4fd2-bab0-9d2411990f76","Type":"ContainerStarted","Data":"60c604afebdb1612d647d47b15a204fe6a06a1e4f5e2efedfd0f847cf6c90b93"}
Jul 24 20:15:56.238572 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.238541    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9t6cs" podStartSLOduration=3.40654477 podStartE2EDuration="22.238530829s" totalImagesPullingTime="18.831986059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:56.238490612 +0000 UTC m=+22.592067769" watchObservedRunningTime="2026-07-24 20:15:56.238530829 +0000 UTC m=+22.592107968"
Jul 24 20:15:56.258499 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.258451    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s9b8j" podStartSLOduration=3.383163851 podStartE2EDuration="22.25844003s" totalImagesPullingTime="18.875276179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:56.258271286 +0000 UTC m=+22.611848426" watchObservedRunningTime="2026-07-24 20:15:56.25844003 +0000 UTC m=+22.612017169"
Jul 24 20:15:56.275348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.275312    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n5gpr" podStartSLOduration=11.607805213 podStartE2EDuration="22.275303365s" totalImagesPullingTime="10.667498152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:56.274930564 +0000 UTC m=+22.628507704" watchObservedRunningTime="2026-07-24 20:15:56.275303365 +0000 UTC m=+22.628880504"
Jul 24 20:15:56.291366 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.291333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t7v97" podStartSLOduration=3.449507504 podStartE2EDuration="22.2913223s" totalImagesPullingTime="18.841814796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:56.291160323 +0000 UTC m=+22.644737462" watchObservedRunningTime="2026-07-24 20:15:56.2913223 +0000 UTC m=+22.644899439"
Jul 24 20:15:56.306040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.306009    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4rcfk" podStartSLOduration=3.4624048 podStartE2EDuration="22.305999197s" totalImagesPullingTime="18.843594397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:56.305976282 +0000 UTC m=+22.659553421" watchObservedRunningTime="2026-07-24 20:15:56.305999197 +0000 UTC m=+22.659576337"
Jul 24 20:15:56.345284 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.345124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:56.441889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.441861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:56.535394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.535370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:56.635036 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:56.635003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.001036 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.000994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.026017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.025988    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 20:15:57.050195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.050122    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T20:15:57.026010891Z","UUID":"c2a36855-e188-48a5-990d-c8acb2c08d63","Handler":null,"Name":"","Endpoint":""}
Jul 24 20:15:57.052056 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.051884    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 20:15:57.052056 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.051927    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 20:15:57.102085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.102049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.146364 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.146336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:57.146500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.146370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:57.146500 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:57.146446    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:57.146637 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:57.146616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:57.241048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.241014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"32b97a060b5f2ede761c62c3d9edebeee0276df8e8829c52ffbd72da898ae0c9"}
Jul 24 20:15:57.241818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.241087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"d29021e64204152228f97f408ec29050f25d0d32d08b402929edb2b7fcac464d"}
Jul 24 20:15:57.241818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.241124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"50cf47af1502508befe23448b73a3dfc7c54cf46bb35ef913c78b8585124b179"}
Jul 24 20:15:57.241818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.241173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"959928ff67a25baa4f8505615ac6b60d6ae49a6d2d669f238626656cefd15050"}
Jul 24 20:15:57.242924 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.242895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4" event={"ID":"9dd76d27-beeb-4fd2-bab0-9d2411990f76","Type":"ContainerStarted","Data":"ddd8a2cbe40f011f8b8a566fbcad4524de79b19c1d16ee28f38ac7ffdbf638fd"}
Jul 24 20:15:57.247412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.247386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p9rgq" event={"ID":"e9654e1a-3b99-487d-9d8f-e1bba3e986e1","Type":"ContainerStarted","Data":"b6078e7fb61fa024f5f5967b00e1e519c6ea81f984134c0f74a6c9a824c688b1"}
Jul 24 20:15:57.398489 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.398382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.481945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.481910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.535926 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.535892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.685911 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.685840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:57.686397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.686376    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:57.708193 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.708143    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p9rgq" podStartSLOduration=4.863476325 podStartE2EDuration="23.708123595s" totalImagesPullingTime="18.84464727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:57.262466094 +0000 UTC m=+23.616043238" watchObservedRunningTime="2026-07-24 20:15:57.708123595 +0000 UTC m=+24.061700738"
Jul 24 20:15:57.717622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.717551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:57.779079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:57.779046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:58.146164 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:58.146136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:15:58.146347 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:58.146274    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:15:58.250182 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:58.250145    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:58.251039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:58.251019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n5gpr"
Jul 24 20:15:58.316562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:58.316536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:59.023148 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.023112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:59.099095 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.099067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:59.146247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.146218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:15:59.146421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.146265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:15:59.146421 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:59.146348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:15:59.146543 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:15:59.146461    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:15:59.200100 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.200070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:15:59.244752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.244724    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rcfk_ad07c18d-61bb-43c7-9e8c-08506c9968ad/dns-node-resolver/0.log"
Jul 24 20:15:59.256647 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.256617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"6ea3a6b864991f77156a67e79ccf1cd05f9e3ee51362e63a65b7f75ba4f4cf8d"}
Jul 24 20:15:59.258887 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.258860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4" event={"ID":"9dd76d27-beeb-4fd2-bab0-9d2411990f76","Type":"ContainerStarted","Data":"4982098d4c1bf45e0d578ed6f801278dc9b9714653f4be534ca4baa58b444e1f"}
Jul 24 20:15:59.278440 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.278357    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kctv4" podStartSLOduration=4.352668779 podStartE2EDuration="25.278341693s" totalImagesPullingTime="20.925672914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:15:59.277790318 +0000 UTC m=+25.631367463" watchObservedRunningTime="2026-07-24 20:15:59.278341693 +0000 UTC m=+25.631918833"
Jul 24 20:15:59.349657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:15:59.349628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:00.146869 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:00.146837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:00.147038 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:00.146952    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:00.171937 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:00.171902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:00.247656 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:00.247631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9t6cs_a6f0789c-a141-42b5-a35f-5d61f8e2188e/node-ca/0.log"
Jul 24 20:16:00.298299 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:00.298258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:00.513377 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:00.513349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:01.146891 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.146702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:01.147047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.146729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:01.147047 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:01.146978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:01.147047 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:01.147029    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:01.267198 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.267163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" event={"ID":"6180745c-de33-463c-834f-fb1c7abde2de","Type":"ContainerStarted","Data":"38e5aaa61f00ab5a5d9fbb5a3183338527f4f583459e721f243d5ab64551a021"}
Jul 24 20:16:01.267394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.267366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:01.267515 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.267425    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:01.269073 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.269050    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="97ada9f59a3fc71adbce88865f358ed9f00a7eeafd9ef5ecf59d10d05495b2b2" exitCode=0
Jul 24 20:16:01.269237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.269085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"97ada9f59a3fc71adbce88865f358ed9f00a7eeafd9ef5ecf59d10d05495b2b2"}
Jul 24 20:16:01.284186 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.284140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:01.338799 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.338758    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf" podStartSLOduration=8.279022713 podStartE2EDuration="27.338746244s" totalImagesPullingTime="19.059723531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:01.311479726 +0000 UTC m=+27.665056866" watchObservedRunningTime="2026-07-24 20:16:01.338746244 +0000 UTC m=+27.692323402"
Jul 24 20:16:01.431065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.430915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:01.656409 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.656321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:01.810068 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.810041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:01.900578 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:01.900552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:02.146812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:02.146788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:02.147015 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:02.146901    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:02.271571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:02.271533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:02.286713 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:02.286689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:02.326305 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:02.326267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:02.926993 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:02.926955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:03.146246 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.146211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:03.146424 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.146215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:03.146424 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:03.146338    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:03.146424 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:03.146389    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:03.275434 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.275405    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="74b845b27cd0aedabdaaa44267aa0864fa2e3090f3c2755404c354023700a0d6" exitCode=0
Jul 24 20:16:03.275583 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.275470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"74b845b27cd0aedabdaaa44267aa0864fa2e3090f3c2755404c354023700a0d6"}
Jul 24 20:16:03.535398 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.535321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:03.736011 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.735981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:03.935193 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:03.935106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:04.137918 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:04.137889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:04.147486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:04.147461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:04.147647 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:04.147623    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:04.335740 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:04.335709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:04.439356 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:04.439328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.146375 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.146343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:05.146375 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.146378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:05.146852 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:05.146448    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:05.146852 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:05.146625    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:05.167471 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.167446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.254722 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.254694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.281529 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.281503    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="eb9cb69f7b28a37203383a359b93a42b2b9335c7ce14435f0fad3991f92d0780" exitCode=0
Jul 24 20:16:05.281657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.281555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"eb9cb69f7b28a37203383a359b93a42b2b9335c7ce14435f0fad3991f92d0780"}
Jul 24 20:16:05.340032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.340008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.541782 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.541751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.729338 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.729311    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.926200 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.926103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:05.987455 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:05.987393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:06.130473 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:06.130434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:06.146547 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:06.146524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:06.147013 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:06.146659    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:06.923330 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:06.923297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:07.146750 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.146722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:07.147150 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.146842    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:07.147150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.146908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:07.147150 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.147019    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:07.335198 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.335153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:07.530410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.530377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:07.734773 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.734684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:07.742787 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.742760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:07.742922 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.742864    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 20:16:07.742987 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.742927    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret podName:502e4008-5e5a-4bee-8936-c5f799313c4f nodeName:}" failed. No retries permitted until 2026-07-24 20:16:39.742914841 +0000 UTC m=+66.096491960 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret") pod "global-pull-secret-syncer-nxgsn" (UID: "502e4008-5e5a-4bee-8936-c5f799313c4f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 20:16:07.844042 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.844014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:07.844243 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.844153    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:16:07.844243 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.844209    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs podName:c249c9d6-0606-4b78-9ce7-281e837c2bee nodeName:}" failed. No retries permitted until 2026-07-24 20:16:39.844194465 +0000 UTC m=+66.197771588 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs") pod "network-metrics-daemon-2tlc7" (UID: "c249c9d6-0606-4b78-9ce7-281e837c2bee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 20:16:07.945069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:07.945039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:07.945244 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.945224    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 20:16:07.945307 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.945250    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 20:16:07.945307 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.945263    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lxzt7 for pod openshift-network-diagnostics/network-check-target-m6v27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:16:07.945400 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:07.945327    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7 podName:628a6c80-bedd-4185-b4ef-0b8a45fa3e31 nodeName:}" failed. No retries permitted until 2026-07-24 20:16:39.945308293 +0000 UTC m=+66.298885429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lxzt7" (UniqueName: "kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7") pod "network-check-target-m6v27" (UID: "628a6c80-bedd-4185-b4ef-0b8a45fa3e31") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 20:16:08.136344 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:08.136307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:08.149347 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:08.149317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:08.149753 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:08.149441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:08.327120 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:08.327083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:08.531074 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:08.531038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:09.146785 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:09.146747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:09.146974 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:09.146795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:09.146974 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:09.146900    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:09.147323 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:09.147299    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:09.324426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:09.324399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:09.572899 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:09.572867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:09.919935 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:09.919907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:10.146870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.146836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:10.147027 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:10.146970    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:10.209973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.209893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:10.295221 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.295190    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="104b566f5750f91eab3b7fdf2e57865518a8e910af601fdc17c8812eb2a9ef64" exitCode=0
Jul 24 20:16:10.295410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.295247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"104b566f5750f91eab3b7fdf2e57865518a8e910af601fdc17c8812eb2a9ef64"}
Jul 24 20:16:10.528252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.528220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:10.719842 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:10.719811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:11.146458 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.146421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:11.146671 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:11.146536    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:11.146671 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.146608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:11.146785 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:11.146702    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:11.153112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.153084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:11.300164 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.300129    2577 generic.go:356] "Generic (PLEG): container finished" podID="48e31ee6-5de8-487f-b301-518d5d7b09c3" containerID="df8362718db3601ecd5359b1dd430b9850d88ac53be00538880b2d57cd6c25c1" exitCode=0
Jul 24 20:16:11.300349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.300172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerDied","Data":"df8362718db3601ecd5359b1dd430b9850d88ac53be00538880b2d57cd6c25c1"}
Jul 24 20:16:11.730472 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.730440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:11.871506 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.871476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:11.972539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:11.972509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:12.128307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:12.128272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:12.146297 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:12.146270    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:12.146423 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:12.146392    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:12.304638 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:12.304589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9px48" event={"ID":"48e31ee6-5de8-487f-b301-518d5d7b09c3","Type":"ContainerStarted","Data":"c130b592c7aaa39c44eaee5730a038d060395286c15e71fd55470f6f99047cee"}
Jul 24 20:16:12.330895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:12.330849    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9px48" podStartSLOduration=8.798087387 podStartE2EDuration="38.330835387s" totalImagesPullingTime="29.532748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:12.330284093 +0000 UTC m=+38.683861233" watchObservedRunningTime="2026-07-24 20:16:12.330835387 +0000 UTC m=+38.684412526"
Jul 24 20:16:13.146750 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:13.146717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:13.146750 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:13.146758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:13.147132 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:13.146846    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:13.147132 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:13.146962    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:14.146791 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:14.146758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:14.147246 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:14.146838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:15.146021 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:15.145993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:15.146021 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:15.146023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:15.146209 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:15.146091    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:15.146243 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:15.146208    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:16.146540 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:16.146511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:16.146927 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:16.146631    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:17.146171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:17.146136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:17.146348 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:17.146244    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:17.146348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:17.146296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:17.146423 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:17.146379    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:18.146403 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:18.146371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:18.146800 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:18.146486    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:18.796255 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:18.796220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:18.862301 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:18.862272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:19.146134 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:19.146045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:19.146302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:19.146045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:19.146302 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:19.146169    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:19.146302 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:19.146227    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:20.146178 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:20.146148    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:20.146560 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:20.146250    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:21.146203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:21.146171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:21.146669 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:21.146273    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:21.146669 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:21.146338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:21.146669 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:21.146445    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:22.145920 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:22.145886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:22.146106 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:22.145984    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:23.146615 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:23.146569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:23.146973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:23.146573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:23.146973 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:23.146673    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:23.146973 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:23.146752    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:24.149165 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:24.149137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:24.149620 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:24.149228    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:24.705356 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:24.705323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:24.812431 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:24.812401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:24.938748 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:24.938719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:25.146217 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:25.146191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:25.146394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:25.146188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:25.146394 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:25.146322    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:25.146394 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:25.146355    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:26.040850 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.040583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:26.045817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.045787    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nxgsn"]
Jul 24 20:16:26.045945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.045898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:26.046034 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:26.046011    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:26.053519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.053453    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6v27"]
Jul 24 20:16:26.053666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.053638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:26.053784 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:26.053750    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:26.054116 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.054099    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tlc7"]
Jul 24 20:16:26.054194 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.054180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:26.054292 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:26.054274    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:26.127114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.127082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:26.596039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.596011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:26.694646 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.694618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:26.770065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:26.770040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:27.057891 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:27.057861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:27.146363 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:27.146333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:27.146506 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:27.146449    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:27.321541 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:27.321460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:27.377571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:27.377540    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:28.146922 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:28.146847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:28.147357 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:28.146847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:28.147357 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:28.147018    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nxgsn" podUID="502e4008-5e5a-4bee-8936-c5f799313c4f"
Jul 24 20:16:28.147357 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:28.146937    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v27" podUID="628a6c80-bedd-4185-b4ef-0b8a45fa3e31"
Jul 24 20:16:28.909511 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:28.909470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.008803 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.008772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.146666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.146636    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:29.146815 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:29.146739    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tlc7" podUID="c249c9d6-0606-4b78-9ce7-281e837c2bee"
Jul 24 20:16:29.223844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.223763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.323140 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.323110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.437660 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.437638    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-110.ec2.internal" event="NodeReady"
Jul 24 20:16:29.437790 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.437752    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 20:16:29.496063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.495996    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5qxkc"]
Jul 24 20:16:29.497847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.497831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.499916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.499895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pxsq4"]
Jul 24 20:16:29.500498 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.500478    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 20:16:29.500498 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.500489    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mgpkh\""
Jul 24 20:16:29.500643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.500584    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 20:16:29.501634 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.501620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.505661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.505340    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 20:16:29.505661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.505443    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 20:16:29.505779 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.505681    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 20:16:29.505779 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.505735    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p92ks\""
Jul 24 20:16:29.510235 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.510214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5qxkc"]
Jul 24 20:16:29.513085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.513063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pxsq4"]
Jul 24 20:16:29.522419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.522401    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gccc9"]
Jul 24 20:16:29.526627 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.526586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.529099 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.529079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 20:16:29.529187 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.529137    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 20:16:29.529646 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.529630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 20:16:29.529722 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.529695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5d6mv\""
Jul 24 20:16:29.529976 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.529963    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 20:16:29.537093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.537074    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gccc9"]
Jul 24 20:16:29.597932 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.597906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mhpw\" (UniqueName: \"kubernetes.io/projected/a149787f-5b57-4218-ac7b-bdd0255538d6-kube-api-access-7mhpw\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.598083 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.597942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-tmp-dir\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.598083 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.597969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-config-volume\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.598083 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.598007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xngzx\" (UniqueName: \"kubernetes.io/projected/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-kube-api-access-xngzx\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.598083 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.598049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-metrics-tls\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.598251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.598115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a149787f-5b57-4218-ac7b-bdd0255538d6-cert\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.642685 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.642652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.698721 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a149787f-5b57-4218-ac7b-bdd0255538d6-cert\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.698822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mhpw\" (UniqueName: \"kubernetes.io/projected/a149787f-5b57-4218-ac7b-bdd0255538d6-kube-api-access-7mhpw\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.698822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-tmp-dir\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.698822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-config-volume\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.698822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xngzx\" (UniqueName: \"kubernetes.io/projected/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-kube-api-access-xngzx\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.698822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zwj\" (UniqueName: \"kubernetes.io/projected/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-api-access-t5zwj\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.699065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-data-volume\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.699065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-metrics-tls\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.699065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.698869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-crio-socket\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.699168 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.699068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.699168 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.699118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-tmp-dir\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.699168 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.699136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.699396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.699377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-config-volume\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.702834 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.702814    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-metrics-tls\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.702949 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.702930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a149787f-5b57-4218-ac7b-bdd0255538d6-cert\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.708354 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.708311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xngzx\" (UniqueName: \"kubernetes.io/projected/8bc0e3a2-3801-4d91-a2fe-075fa215a0c7-kube-api-access-xngzx\") pod \"dns-default-5qxkc\" (UID: \"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7\") " pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.708469 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.708375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mhpw\" (UniqueName: \"kubernetes.io/projected/a149787f-5b57-4218-ac7b-bdd0255538d6-kube-api-access-7mhpw\") pod \"ingress-canary-pxsq4\" (UID: \"a149787f-5b57-4218-ac7b-bdd0255538d6\") " pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.712703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.712665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.784469 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.784447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.799581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zwj\" (UniqueName: \"kubernetes.io/projected/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-api-access-t5zwj\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799658 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-data-volume\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799658 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-crio-socket\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799900 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-data-volume\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.799936 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.799892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-crio-socket\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.800229 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.800204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.801909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.801888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.808994 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.808974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:29.811323 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.811300    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zwj\" (UniqueName: \"kubernetes.io/projected/f59d6d58-0fc5-4ee4-9239-25b25682cbb4-kube-api-access-t5zwj\") pod \"insights-runtime-extractor-gccc9\" (UID: \"f59d6d58-0fc5-4ee4-9239-25b25682cbb4\") " pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.814534 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.814519    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pxsq4"
Jul 24 20:16:29.827848 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.827825    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26f5w"]
Jul 24 20:16:29.829653 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.829635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:29.834029 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834011    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 20:16:29.834135 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5npz6\""
Jul 24 20:16:29.834135 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834067    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 20:16:29.834252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gccc9"
Jul 24 20:16:29.834552 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834535    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 20:16:29.834725 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834656    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 20:16:29.834725 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.834668    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 20:16:29.844417 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.844242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26f5w"]
Jul 24 20:16:29.881275 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.881162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:29.965989 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.965954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5qxkc"]
Jul 24 20:16:29.968586 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:29.968561    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc0e3a2_3801_4d91_a2fe_075fa215a0c7.slice/crio-707f6dc4468399f028225b331c92e2fcb6f0d5efad292b209d94aa943ebaeae2 WatchSource:0}: Error finding container 707f6dc4468399f028225b331c92e2fcb6f0d5efad292b209d94aa943ebaeae2: Status 404 returned error can't find the container with id 707f6dc4468399f028225b331c92e2fcb6f0d5efad292b209d94aa943ebaeae2
Jul 24 20:16:29.981241 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.981199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pxsq4"]
Jul 24 20:16:29.985278 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:29.985255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda149787f_5b57_4218_ac7b_bdd0255538d6.slice/crio-ef5a217448120d835fe942fa09f50f0c0db148941ea0494d6383bdb373cd1eba WatchSource:0}: Error finding container ef5a217448120d835fe942fa09f50f0c0db148941ea0494d6383bdb373cd1eba: Status 404 returned error can't find the container with id ef5a217448120d835fe942fa09f50f0c0db148941ea0494d6383bdb373cd1eba
Jul 24 20:16:29.998371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:29.998349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gccc9"]
Jul 24 20:16:30.001030 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.001007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqgkc\" (UniqueName: \"kubernetes.io/projected/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-kube-api-access-wqgkc\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.001142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.001080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.001142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.001122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.001250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.001147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.001250 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:30.001173    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59d6d58_0fc5_4ee4_9239_25b25682cbb4.slice/crio-50634faa22c8a92eb93f2e53ab7e83648ab04a0a1df875b3bfaf88197f62e98e WatchSource:0}: Error finding container 50634faa22c8a92eb93f2e53ab7e83648ab04a0a1df875b3bfaf88197f62e98e: Status 404 returned error can't find the container with id 50634faa22c8a92eb93f2e53ab7e83648ab04a0a1df875b3bfaf88197f62e98e
Jul 24 20:16:30.053134 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.053050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:30.102434 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.102406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgkc\" (UniqueName: \"kubernetes.io/projected/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-kube-api-access-wqgkc\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.102564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.102480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.102564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.102527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.102564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.102552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.103232 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.103210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.105126 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.105105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.105199 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.105168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.110671 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.110652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgkc\" (UniqueName: \"kubernetes.io/projected/bd4cc4db-5d53-4a79-8bfe-8c39d015a7da-kube-api-access-wqgkc\") pod \"prometheus-operator-84d69f794d-26f5w\" (UID: \"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.140518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.140496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w"
Jul 24 20:16:30.149437 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.149423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:30.149650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.149630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:30.152226 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.152204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 20:16:30.152317 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.152245    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 20:16:30.152317 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.152260    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8xdt\""
Jul 24 20:16:30.152418 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.152384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 20:16:30.259876 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.259846    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-26f5w"]
Jul 24 20:16:30.263546 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:30.263519    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4cc4db_5d53_4a79_8bfe_8c39d015a7da.slice/crio-719761df8e82b8aadfce78330a6e9d17aa7dc5ee6b5ac5775c2deee24aa02621 WatchSource:0}: Error finding container 719761df8e82b8aadfce78330a6e9d17aa7dc5ee6b5ac5775c2deee24aa02621: Status 404 returned error can't find the container with id 719761df8e82b8aadfce78330a6e9d17aa7dc5ee6b5ac5775c2deee24aa02621
Jul 24 20:16:30.346232 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.346159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w" event={"ID":"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da","Type":"ContainerStarted","Data":"719761df8e82b8aadfce78330a6e9d17aa7dc5ee6b5ac5775c2deee24aa02621"}
Jul 24 20:16:30.347176 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.347155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gccc9" event={"ID":"f59d6d58-0fc5-4ee4-9239-25b25682cbb4","Type":"ContainerStarted","Data":"50634faa22c8a92eb93f2e53ab7e83648ab04a0a1df875b3bfaf88197f62e98e"}
Jul 24 20:16:30.348162 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.348137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxkc" event={"ID":"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7","Type":"ContainerStarted","Data":"707f6dc4468399f028225b331c92e2fcb6f0d5efad292b209d94aa943ebaeae2"}
Jul 24 20:16:30.349103 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:30.349084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pxsq4" event={"ID":"a149787f-5b57-4218-ac7b-bdd0255538d6","Type":"ContainerStarted","Data":"ef5a217448120d835fe942fa09f50f0c0db148941ea0494d6383bdb373cd1eba"}
Jul 24 20:16:31.146561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:31.146526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:31.151618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:31.151192    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sm2r\""
Jul 24 20:16:31.151618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:31.151438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 20:16:33.337672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:33.337639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:33.393969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:33.393936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:33.676699 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:33.676624    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:33.732172 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:33.732139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:34.296497 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:34.296466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hxdhf"
Jul 24 20:16:34.471273 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:34.471233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:34.605633 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:34.605537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:36.716486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:36.716453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:37.202617 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.202562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:37.320500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.320439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:37.373711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.373674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w" event={"ID":"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da","Type":"ContainerStarted","Data":"bbd842eb2cf7b3bb7de3c786c0aea92aaee516a9d7e8c8c01081ccb551d82be2"}
Jul 24 20:16:37.373884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.373742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w" event={"ID":"bd4cc4db-5d53-4a79-8bfe-8c39d015a7da","Type":"ContainerStarted","Data":"e6dd65a568a35bc9558aea47f8c38010c296d96ffc7de9876588381445e70c9f"}
Jul 24 20:16:37.375916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.375875    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gccc9" event={"ID":"f59d6d58-0fc5-4ee4-9239-25b25682cbb4","Type":"ContainerStarted","Data":"8164e598c0adf3bf8c69611d8d48f7e6df73268b0b333e65639b76603f6a659c"}
Jul 24 20:16:37.378209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.378096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxkc" event={"ID":"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7","Type":"ContainerStarted","Data":"d9284eca5b03584f63ed4183bd02cc3089860ac0dec7655097c8d99c9c15d6b4"}
Jul 24 20:16:37.378209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.378143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5qxkc" event={"ID":"8bc0e3a2-3801-4d91-a2fe-075fa215a0c7","Type":"ContainerStarted","Data":"520e2efc1cfba7bb13c751ad3747da7c7a948a97e3bb12dc087732927c3d1fe3"}
Jul 24 20:16:37.378209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.378175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:37.379659 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.379635    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pxsq4" event={"ID":"a149787f-5b57-4218-ac7b-bdd0255538d6","Type":"ContainerStarted","Data":"3a97be1285529de1a8f7f1a6a84ecb85b7163ec8dc0944f253af89b6a37b412a"}
Jul 24 20:16:37.400033 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.399972    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-26f5w" podStartSLOduration=2.73091948 podStartE2EDuration="8.399954294s" totalImagesPullingTime="5.669034814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:37.399737715 +0000 UTC m=+63.753314882" watchObservedRunningTime="2026-07-24 20:16:37.399954294 +0000 UTC m=+63.753531413"
Jul 24 20:16:37.438582 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:37.438536    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5qxkc" podStartSLOduration=2.474315644 podStartE2EDuration="8.438518303s" totalImagesPullingTime="5.964202659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:37.437794293 +0000 UTC m=+63.791371450" watchObservedRunningTime="2026-07-24 20:16:37.438518303 +0000 UTC m=+63.792095442"
Jul 24 20:16:38.707731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:38.707690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:38.884849 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:38.884810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.050103 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.050068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.147049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.146965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.248732 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.248696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.315823 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.315726    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pxsq4" podStartSLOduration=4.0452801 podStartE2EDuration="10.315708374s" totalImagesPullingTime="6.270428274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:37.469020684 +0000 UTC m=+63.822597822" watchObservedRunningTime="2026-07-24 20:16:39.315708374 +0000 UTC m=+65.669285560"
Jul 24 20:16:39.315987 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.315935    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"]
Jul 24 20:16:39.323579 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.323557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.326279 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.326257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 20:16:39.327013 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.326962    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 20:16:39.327140 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.326970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dnbsg\""
Jul 24 20:16:39.329201 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.329176    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"]
Jul 24 20:16:39.333666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.333213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.333898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.333880    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"]
Jul 24 20:16:39.336897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.336875    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 20:16:39.337062 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.337036    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 20:16:39.337155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.337139    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cdkgr\""
Jul 24 20:16:39.337325 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.337312    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 20:16:39.350741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.350712    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dwjmk"]
Jul 24 20:16:39.353794 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.353774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.363968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.363945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.367918 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.367900    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 20:16:39.368023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.367960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 20:16:39.368100 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.368078    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 20:16:39.368188 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.368150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p7mnv\""
Jul 24 20:16:39.376360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-metrics-client-ca\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376479 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.376479 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-textfile\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376479 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.376620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.376620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnztp\" (UniqueName: \"kubernetes.io/projected/5bd7b2fa-1edb-4163-8855-967b3ded1df6-kube-api-access-tnztp\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.376620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-sys\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376775 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-wtmp\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376775 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.376775 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx8v6\" (UniqueName: \"kubernetes.io/projected/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-kube-api-access-fx8v6\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376775 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd7b2fa-1edb-4163-8855-967b3ded1df6-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsjdv\" (UniqueName: \"kubernetes.io/projected/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-api-access-zsjdv\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-root\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-tls\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.376916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.377125 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.376930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c90c70df-5a5c-43a2-9bdd-853a87651a08-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.392996 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.392971    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"]
Jul 24 20:16:39.477471 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-metrics-client-ca\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.477666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-textfile\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.477666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.477666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnztp\" (UniqueName: \"kubernetes.io/projected/5bd7b2fa-1edb-4163-8855-967b3ded1df6-kube-api-access-tnztp\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-sys\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-wtmp\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx8v6\" (UniqueName: \"kubernetes.io/projected/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-kube-api-access-fx8v6\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd7b2fa-1edb-4163-8855-967b3ded1df6-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjdv\" (UniqueName: \"kubernetes.io/projected/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-api-access-zsjdv\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.477933 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.477931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-root\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-wtmp\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-sys\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-tls\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:39.478250    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:39.478303    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls podName:5bd7b2fa-1edb-4163-8855-967b3ded1df6 nodeName:}" failed. No retries permitted until 2026-07-24 20:16:39.978283798 +0000 UTC m=+66.331860933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-dvmg2" (UID: "5bd7b2fa-1edb-4163-8855-967b3ded1df6") : secret "openshift-state-metrics-tls" not found
Jul 24 20:16:39.478350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c90c70df-5a5c-43a2-9bdd-853a87651a08-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.478793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.478793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-metrics-client-ca\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-root\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.478793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c90c70df-5a5c-43a2-9bdd-853a87651a08-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.478793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bd7b2fa-1edb-4163-8855-967b3ded1df6-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.479044 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:39.478823    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 20:16:39.479044 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:39.478881    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls podName:c90c70df-5a5c-43a2-9bdd-853a87651a08 nodeName:}" failed. No retries permitted until 2026-07-24 20:16:39.978864651 +0000 UTC m=+66.332441784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-k9shx" (UID: "c90c70df-5a5c-43a2-9bdd-853a87651a08") : secret "kube-state-metrics-tls" not found
Jul 24 20:16:39.479044 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.478944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90c70df-5a5c-43a2-9bdd-853a87651a08-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.479206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.479172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-textfile\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.479206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.479196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.481562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.481535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-tls\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.481562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.481567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.481782 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.481648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.482168 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.482146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.493759 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.493732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjdv\" (UniqueName: \"kubernetes.io/projected/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-api-access-zsjdv\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.494061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.494046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx8v6\" (UniqueName: \"kubernetes.io/projected/86b45308-92d0-450f-bb8d-4ec2f4dd12cc-kube-api-access-fx8v6\") pod \"node-exporter-dwjmk\" (UID: \"86b45308-92d0-450f-bb8d-4ec2f4dd12cc\") " pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.495411 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.495389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnztp\" (UniqueName: \"kubernetes.io/projected/5bd7b2fa-1edb-4163-8855-967b3ded1df6-kube-api-access-tnztp\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.519661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.519623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.618303 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.618210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.662409 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.662375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwjmk"
Jul 24 20:16:39.695086 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:39.695057    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b45308_92d0_450f_bb8d_4ec2f4dd12cc.slice/crio-c6dc9a7a7205a9970670c0998dde3bf621a4ca008b550c6a20536bcd2d3ca786 WatchSource:0}: Error finding container c6dc9a7a7205a9970670c0998dde3bf621a4ca008b550c6a20536bcd2d3ca786: Status 404 returned error can't find the container with id c6dc9a7a7205a9970670c0998dde3bf621a4ca008b550c6a20536bcd2d3ca786
Jul 24 20:16:39.714997 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.714965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.781256 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.781229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:39.783784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.783765    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 20:16:39.794123 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.794101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/502e4008-5e5a-4bee-8936-c5f799313c4f-original-pull-secret\") pod \"global-pull-secret-syncer-nxgsn\" (UID: \"502e4008-5e5a-4bee-8936-c5f799313c4f\") " pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:39.881973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.881901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:39.884658 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.884641    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 20:16:39.894541 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.894513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c249c9d6-0606-4b78-9ce7-281e837c2bee-metrics-certs\") pod \"network-metrics-daemon-2tlc7\" (UID: \"c249c9d6-0606-4b78-9ce7-281e837c2bee\") " pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:39.982573 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.982537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:39.982726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.982584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.982792 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.982758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.985060 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.985036    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 20:16:39.985452 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.985432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c90c70df-5a5c-43a2-9bdd-853a87651a08-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-k9shx\" (UID: \"c90c70df-5a5c-43a2-9bdd-853a87651a08\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:39.985678 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.985656    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bd7b2fa-1edb-4163-8855-967b3ded1df6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-dvmg2\" (UID: \"5bd7b2fa-1edb-4163-8855-967b3ded1df6\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:39.988846 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.988818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:39.996296 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:39.996279    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 20:16:40.007336 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.007311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxzt7\" (UniqueName: \"kubernetes.io/projected/628a6c80-bedd-4185-b4ef-0b8a45fa3e31-kube-api-access-lxzt7\") pod \"network-check-target-m6v27\" (UID: \"628a6c80-bedd-4185-b4ef-0b8a45fa3e31\") " pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:40.061098 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.061063    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b8xdt\""
Jul 24 20:16:40.063550 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.063529    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nxgsn"
Jul 24 20:16:40.069433 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.069404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:40.165802 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.165570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2sm2r\""
Jul 24 20:16:40.173106 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.172997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tlc7"
Jul 24 20:16:40.226501 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.226466    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nxgsn"]
Jul 24 20:16:40.233176 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:40.233149    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502e4008_5e5a_4bee_8936_c5f799313c4f.slice/crio-3c9eca6ea3f6dc393337812ca4a480651fd1f10a65f6598ca7877ca0e3a4d3be WatchSource:0}: Error finding container 3c9eca6ea3f6dc393337812ca4a480651fd1f10a65f6598ca7877ca0e3a4d3be: Status 404 returned error can't find the container with id 3c9eca6ea3f6dc393337812ca4a480651fd1f10a65f6598ca7877ca0e3a4d3be
Jul 24 20:16:40.236949 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.236870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"
Jul 24 20:16:40.244173 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.244107    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6v27"]
Jul 24 20:16:40.244843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.244816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"
Jul 24 20:16:40.254063 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:40.254033    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628a6c80_bedd_4185_b4ef_0b8a45fa3e31.slice/crio-43427e23f004d5564951625987b4e25317d12f50c75273618ca5855c8e264668 WatchSource:0}: Error finding container 43427e23f004d5564951625987b4e25317d12f50c75273618ca5855c8e264668: Status 404 returned error can't find the container with id 43427e23f004d5564951625987b4e25317d12f50c75273618ca5855c8e264668
Jul 24 20:16:40.265031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.264954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:40.349705 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.349671    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tlc7"]
Jul 24 20:16:40.394197 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.394154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjmk" event={"ID":"86b45308-92d0-450f-bb8d-4ec2f4dd12cc","Type":"ContainerStarted","Data":"c6dc9a7a7205a9970670c0998dde3bf621a4ca008b550c6a20536bcd2d3ca786"}
Jul 24 20:16:40.397367 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.397339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gccc9" event={"ID":"f59d6d58-0fc5-4ee4-9239-25b25682cbb4","Type":"ContainerStarted","Data":"bec35dc300b47107caa51145b801896087fc029ce9379d806d8dc3294f07a824"}
Jul 24 20:16:40.398670 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.398642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6v27" event={"ID":"628a6c80-bedd-4185-b4ef-0b8a45fa3e31","Type":"ContainerStarted","Data":"43427e23f004d5564951625987b4e25317d12f50c75273618ca5855c8e264668"}
Jul 24 20:16:40.400065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.400023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nxgsn" event={"ID":"502e4008-5e5a-4bee-8936-c5f799313c4f","Type":"ContainerStarted","Data":"3c9eca6ea3f6dc393337812ca4a480651fd1f10a65f6598ca7877ca0e3a4d3be"}
Jul 24 20:16:40.410651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.409665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:40.414656 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.413585    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:16:40.426185 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.426160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.433252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.433222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 20:16:40.437571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.436578    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 20:16:40.437571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.436734    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 20:16:40.437571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.436861    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 20:16:40.437571 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.436979    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 20:16:40.439159 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.438896    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f9j4l\""
Jul 24 20:16:40.440697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.439744    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 20:16:40.440697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.439958    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 20:16:40.440697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.440205    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 20:16:40.440697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.440507    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 20:16:40.440697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.440682    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2"]
Jul 24 20:16:40.441894 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.441874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:16:40.460171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.460148    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx"]
Jul 24 20:16:40.487736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.487910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.487910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.487910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.487910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.487910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc59s\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.487971    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.488006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.488046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.488079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.488111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.488174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.488140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc59s\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.589527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:40.590069    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:40.590111    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle podName:3934fd0f-8bfd-4ade-b4bd-f810e282f12b nodeName:}" failed. No retries permitted until 2026-07-24 20:16:41.090086501 +0000 UTC m=+67.443663658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b") : configmap references non-existent config key: ca-bundle.crt
Jul 24 20:16:40.591400 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:40.590139    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls podName:3934fd0f-8bfd-4ade-b4bd-f810e282f12b nodeName:}" failed. No retries permitted until 2026-07-24 20:16:41.090129005 +0000 UTC m=+67.443706126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b") : secret "alertmanager-main-tls" not found
Jul 24 20:16:40.592332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.590261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.592332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.590922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.593151 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.593123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.593866 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.593841    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.593866 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.593854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.595060 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.595035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.595788 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.595313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.595788 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.595748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.596310 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.596282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.597581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.597559    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.606749 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.606726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc59s\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:40.712518 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:40.712440    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc249c9d6_0606_4b78_9ce7_281e837c2bee.slice/crio-b2dbf7fcbdf3585735914bdf766bf119ce4caa1b00830c139e640f1d1a1d7c3f WatchSource:0}: Error finding container b2dbf7fcbdf3585735914bdf766bf119ce4caa1b00830c139e640f1d1a1d7c3f: Status 404 returned error can't find the container with id b2dbf7fcbdf3585735914bdf766bf119ce4caa1b00830c139e640f1d1a1d7c3f
Jul 24 20:16:40.713444 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:40.713334    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90c70df_5a5c_43a2_9bdd_853a87651a08.slice/crio-a3c34716c5c1d0900bdc49a4579fc96f4bb2e740539c807ad1d04bebe344706c WatchSource:0}: Error finding container a3c34716c5c1d0900bdc49a4579fc96f4bb2e740539c807ad1d04bebe344706c: Status 404 returned error can't find the container with id a3c34716c5c1d0900bdc49a4579fc96f4bb2e740539c807ad1d04bebe344706c
Jul 24 20:16:40.714422 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:40.714304    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd7b2fa_1edb_4163_8855_967b3ded1df6.slice/crio-5dd37e62468fb7580ca766a48c523df976504158bd92a25661f4fea1a082f66e WatchSource:0}: Error finding container 5dd37e62468fb7580ca766a48c523df976504158bd92a25661f4fea1a082f66e: Status 404 returned error can't find the container with id 5dd37e62468fb7580ca766a48c523df976504158bd92a25661f4fea1a082f66e
Jul 24 20:16:40.724328 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.724303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:40.819371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.819277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:40.933154 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:40.933080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:41.093764 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.093723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:41.093950 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.093783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:41.094688 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.094659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:41.096879 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.096853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:41.269433 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.269085    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b688554b-ghxn9"]
Jul 24 20:16:41.296698 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.295630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b688554b-ghxn9"]
Jul 24 20:16:41.296698 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.296084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.299878 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.299843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 20:16:41.300031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.299964    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-65g8roi4oc3o7\""
Jul 24 20:16:41.300114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.300070    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 20:16:41.300114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.300103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 20:16:41.301471 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.301254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 20:16:41.301471 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.301303    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j9zlz\""
Jul 24 20:16:41.301691 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.301569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 20:16:41.348165 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.347836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:16:41.397873 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.397837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.398069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.397893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.398069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.397923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-grpc-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.398069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.397958    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.398069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.398000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eed69cd-9878-499b-a9b8-6dd5bd754c27-metrics-client-ca\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.399014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.398100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg7l7\" (UniqueName: \"kubernetes.io/projected/7eed69cd-9878-499b-a9b8-6dd5bd754c27-kube-api-access-mg7l7\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.399014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.398161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.399014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.398200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.409527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.408938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx" event={"ID":"c90c70df-5a5c-43a2-9bdd-853a87651a08","Type":"ContainerStarted","Data":"a3c34716c5c1d0900bdc49a4579fc96f4bb2e740539c807ad1d04bebe344706c"}
Jul 24 20:16:41.412681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.412526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2" event={"ID":"5bd7b2fa-1edb-4163-8855-967b3ded1df6","Type":"ContainerStarted","Data":"360f1a923b4c628af8e6f4a4794f949b6f846706558061747360f8b92e0c0ec0"}
Jul 24 20:16:41.412681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.412588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2" event={"ID":"5bd7b2fa-1edb-4163-8855-967b3ded1df6","Type":"ContainerStarted","Data":"fe140bc5af06421b7aa44fe08d93724c16d558c9e039a1dc17a96c4ec367c2e5"}
Jul 24 20:16:41.412681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.412641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2" event={"ID":"5bd7b2fa-1edb-4163-8855-967b3ded1df6","Type":"ContainerStarted","Data":"5dd37e62468fb7580ca766a48c523df976504158bd92a25661f4fea1a082f66e"}
Jul 24 20:16:41.415422 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.415344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tlc7" event={"ID":"c249c9d6-0606-4b78-9ce7-281e837c2bee","Type":"ContainerStarted","Data":"b2dbf7fcbdf3585735914bdf766bf119ce4caa1b00830c139e640f1d1a1d7c3f"}
Jul 24 20:16:41.419255 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.418518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjmk" event={"ID":"86b45308-92d0-450f-bb8d-4ec2f4dd12cc","Type":"ContainerStarted","Data":"a51e53b1b60d9c71ebc60f59849ed83c4f684c485ab4fa93750faea580073196"}
Jul 24 20:16:41.499151 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eed69cd-9878-499b-a9b8-6dd5bd754c27-metrics-client-ca\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499279 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg7l7\" (UniqueName: \"kubernetes.io/projected/7eed69cd-9878-499b-a9b8-6dd5bd754c27-kube-api-access-mg7l7\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499451 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499451 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499451 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-grpc-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.499614 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.499462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.501061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.501034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eed69cd-9878-499b-a9b8-6dd5bd754c27-metrics-client-ca\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.503438 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.503416    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.504953 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.504908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-grpc-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.517414 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.516866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.518686 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.518585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-tls\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.521255 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.521207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.521570 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.521528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg7l7\" (UniqueName: \"kubernetes.io/projected/7eed69cd-9878-499b-a9b8-6dd5bd754c27-kube-api-access-mg7l7\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.521889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.521865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7eed69cd-9878-499b-a9b8-6dd5bd754c27-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b688554b-ghxn9\" (UID: \"7eed69cd-9878-499b-a9b8-6dd5bd754c27\") " pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.566858 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.566757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:16:41.568786 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:41.567797    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3934fd0f_8bfd_4ade_b4bd_f810e282f12b.slice/crio-e63104b8d77c6b55acfdcf6383ddc43ab60a45e55ffe7ea0ded415305684356c WatchSource:0}: Error finding container e63104b8d77c6b55acfdcf6383ddc43ab60a45e55ffe7ea0ded415305684356c: Status 404 returned error can't find the container with id e63104b8d77c6b55acfdcf6383ddc43ab60a45e55ffe7ea0ded415305684356c
Jul 24 20:16:41.610198 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.609909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:41.819226 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:41.819113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b688554b-ghxn9"]
Jul 24 20:16:42.260283 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.260214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:42.364445 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.364382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:42.427623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.426793    2577 generic.go:356] "Generic (PLEG): container finished" podID="86b45308-92d0-450f-bb8d-4ec2f4dd12cc" containerID="a51e53b1b60d9c71ebc60f59849ed83c4f684c485ab4fa93750faea580073196" exitCode=0
Jul 24 20:16:42.427623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.426870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjmk" event={"ID":"86b45308-92d0-450f-bb8d-4ec2f4dd12cc","Type":"ContainerDied","Data":"a51e53b1b60d9c71ebc60f59849ed83c4f684c485ab4fa93750faea580073196"}
Jul 24 20:16:42.430001 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.429913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"d2d3cbb7799ff218263db030596d44a6485708f8b3182965de523e16d40e52d2"}
Jul 24 20:16:42.432569 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.432472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"e63104b8d77c6b55acfdcf6383ddc43ab60a45e55ffe7ea0ded415305684356c"}
Jul 24 20:16:42.451700 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.451589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:42.551178 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.551084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:42.615326 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.615233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:42.711767 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:42.711727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:43.706243 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.705345    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-588c96548f-5ffz8"]
Jul 24 20:16:43.708030 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.708005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.711040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.711019    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 20:16:43.714040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.712327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 20:16:43.714040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.712534    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1ehphtei9h6sv\""
Jul 24 20:16:43.714040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.712733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 20:16:43.714040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.713037    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 20:16:43.714040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.713268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bvlz2\""
Jul 24 20:16:43.724355 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.724329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-588c96548f-5ffz8"]
Jul 24 20:16:43.823759 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.823729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-audit-log\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.823922 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.823771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-tls\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.823922 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.823878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.824018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.823934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-client-certs\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.824018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.823975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2p27\" (UniqueName: \"kubernetes.io/projected/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-kube-api-access-x2p27\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.824104 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.824023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-metrics-server-audit-profiles\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.824104 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.824051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-client-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.924555 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.924523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.924755 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.924570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-client-certs\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.924935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2p27\" (UniqueName: \"kubernetes.io/projected/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-kube-api-access-x2p27\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.925013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-metrics-server-audit-profiles\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.925049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-client-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.925092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-audit-log\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.925136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-tls\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.925468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.925382    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.926270 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.926246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-metrics-server-audit-profiles\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.927104 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.927078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-audit-log\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.927935 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.927913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-client-certs\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.928371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.928349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-secret-metrics-server-tls\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.929652 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.929631    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-client-ca-bundle\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:43.935922 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:43.935882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2p27\" (UniqueName: \"kubernetes.io/projected/947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5-kube-api-access-x2p27\") pod \"metrics-server-588c96548f-5ffz8\" (UID: \"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5\") " pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:44.015199 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.015135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:44.024484 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.024453    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"]
Jul 24 20:16:44.026219 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.026197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:16:44.033985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.033669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.037541 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.036822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-x6dgr\""
Jul 24 20:16:44.037541 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.037058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 20:16:44.038219 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.038192    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"]
Jul 24 20:16:44.127232 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.127192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz6fq\" (UniqueName: \"kubernetes.io/projected/00535512-56a8-4c9e-b62c-5b9e49dc692f-kube-api-access-lz6fq\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.127406 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.127285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.228396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.228356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz6fq\" (UniqueName: \"kubernetes.io/projected/00535512-56a8-4c9e-b62c-5b9e49dc692f-kube-api-access-lz6fq\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.228574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.228458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.228657 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:44.228583    2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 20:16:44.228720 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:16:44.228671    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert podName:00535512-56a8-4c9e-b62c-5b9e49dc692f nodeName:}" failed. No retries permitted until 2026-07-24 20:16:44.728649536 +0000 UTC m=+71.082226656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-jk45v" (UID: "00535512-56a8-4c9e-b62c-5b9e49dc692f") : secret "monitoring-plugin-cert" not found
Jul 24 20:16:44.239521 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.239495    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz6fq\" (UniqueName: \"kubernetes.io/projected/00535512-56a8-4c9e-b62c-5b9e49dc692f-kube-api-access-lz6fq\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.363152 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.363068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:44.734092 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.734008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.736697 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.736672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00535512-56a8-4c9e-b62c-5b9e49dc692f-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-jk45v\" (UID: \"00535512-56a8-4c9e-b62c-5b9e49dc692f\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:44.948681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:44.948264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:45.433642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.433580    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-8pzq7"]
Jul 24 20:16:45.436105 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.436076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:16:45.439134 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.438896    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4qzrs\""
Jul 24 20:16:45.439134 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.438987    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 20:16:45.439134 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.439027    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 20:16:45.447580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.447492    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-8pzq7"]
Jul 24 20:16:45.454098 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.454074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:45.522832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.522803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:45.539952 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.539919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwjhj\" (UniqueName: \"kubernetes.io/projected/9005214a-0920-4d14-a92b-c9a7e9b20617-kube-api-access-xwjhj\") pod \"downloads-7c97dc4555-8pzq7\" (UID: \"9005214a-0920-4d14-a92b-c9a7e9b20617\") " pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:16:45.617168 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.617129    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:16:45.626631 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.626063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.630553 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.630524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:45.631967 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.631944    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 20:16:45.632115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.632097    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 20:16:45.632503 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.632477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 20:16:45.632652 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.632577    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 20:16:45.633440 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.633419    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 20:16:45.633703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.633686    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 20:16:45.633894 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.633879    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 20:16:45.634114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.634099    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8mab07kva0fbb\""
Jul 24 20:16:45.634307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.634292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m7vxs\""
Jul 24 20:16:45.634363 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.634353    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 20:16:45.634640 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.634622    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 20:16:45.635064 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.635040    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 20:16:45.635502 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.635180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 20:16:45.635579 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.635508    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 20:16:45.635772 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.635098    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 20:16:45.640484 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.640458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwjhj\" (UniqueName: \"kubernetes.io/projected/9005214a-0920-4d14-a92b-c9a7e9b20617-kube-api-access-xwjhj\") pod \"downloads-7c97dc4555-8pzq7\" (UID: \"9005214a-0920-4d14-a92b-c9a7e9b20617\") " pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:16:45.640588 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.640485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 20:16:45.648699 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.648670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:16:45.661001 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.660974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwjhj\" (UniqueName: \"kubernetes.io/projected/9005214a-0920-4d14-a92b-c9a7e9b20617-kube-api-access-xwjhj\") pod \"downloads-7c97dc4555-8pzq7\" (UID: \"9005214a-0920-4d14-a92b-c9a7e9b20617\") " pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:16:45.721058 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.720977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:45.741505 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.741505 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cftwk\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.741996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.742623 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.742568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.750007 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.749984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:16:45.843419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843636 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843636 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843636 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.843804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.843993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.844027 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.844017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.846719 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.844062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.846719 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.844092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftwk\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.846719 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.844117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.847875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.847349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.847875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.847496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.847875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.847659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.847875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.847741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.848135 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.848032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.848190 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.848148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.848753 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.848432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.848753 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.848663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.849944 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.849642    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.850283 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.850263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.850364 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.850268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.850435 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.850400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.850622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.850581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.852381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.852358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.852937 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.852911    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.853024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.852970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.853159 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.853111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.854652 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.854629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.855123 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.855101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.859581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.859554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftwk\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk\") pod \"prometheus-k8s-0\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:45.939085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:45.938855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:46.080295 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:46.080259    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:46.138725 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:46.138682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:46.476840 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:46.476755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:46.581012 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:46.580964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:47.111872 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:47.111835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:47.212160 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:47.212122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:47.311425 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:47.311388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:47.386735 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:47.386656    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5qxkc"
Jul 24 20:16:47.668555 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:47.668452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:48.309852 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.309810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:48.411316 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.411280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:48.730159 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.729803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-588c96548f-5ffz8"]
Jul 24 20:16:48.742687 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:48.742637    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947df3b0_8b1f_404b_a7bd_6fb9d01ba3b5.slice/crio-dd1641996553c6d911592901d8f62af746d58bd8c02de205f658636f40105cb6 WatchSource:0}: Error finding container dd1641996553c6d911592901d8f62af746d58bd8c02de205f658636f40105cb6: Status 404 returned error can't find the container with id dd1641996553c6d911592901d8f62af746d58bd8c02de205f658636f40105cb6
Jul 24 20:16:48.769567 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.769487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"]
Jul 24 20:16:48.792410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.792382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-8pzq7"]
Jul 24 20:16:48.807197 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.807151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:48.816044 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:48.815833    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9005214a_0920_4d14_a92b_c9a7e9b20617.slice/crio-fb10b59e6a6a4c72d945143c5dd707ad2da046b1f58ea2363c56c6a534f74e3c WatchSource:0}: Error finding container fb10b59e6a6a4c72d945143c5dd707ad2da046b1f58ea2363c56c6a534f74e3c: Status 404 returned error can't find the container with id fb10b59e6a6a4c72d945143c5dd707ad2da046b1f58ea2363c56c6a534f74e3c
Jul 24 20:16:48.866499 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.863244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:16:48.879763 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:16:48.876695    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ce60cd_7b93_441b_864d_66002c82126c.slice/crio-6bd971e2af3588a3229bebdc441ebde9185b41e0db37ae4f5e0dddc607ad3d33 WatchSource:0}: Error finding container 6bd971e2af3588a3229bebdc441ebde9185b41e0db37ae4f5e0dddc607ad3d33: Status 404 returned error can't find the container with id 6bd971e2af3588a3229bebdc441ebde9185b41e0db37ae4f5e0dddc607ad3d33
Jul 24 20:16:48.904195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:48.904127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:49.473412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.472477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gccc9" event={"ID":"f59d6d58-0fc5-4ee4-9239-25b25682cbb4","Type":"ContainerStarted","Data":"f237f363b24304f328dd74510aee0386bcfc37a5fa28836206e06ccf844da98b"}
Jul 24 20:16:49.475797 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.475738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6v27" event={"ID":"628a6c80-bedd-4185-b4ef-0b8a45fa3e31","Type":"ContainerStarted","Data":"2e6f2e7a8d441f9861895100e6e0f47d86ab688961b522dffff177b897568286"}
Jul 24 20:16:49.475946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.475932    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:16:49.479339 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.479223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx" event={"ID":"c90c70df-5a5c-43a2-9bdd-853a87651a08","Type":"ContainerStarted","Data":"1dbc3c03cd0cef9d1daf81d9d14d982ca8a392cbc62c10685b83c1a2499e78f8"}
Jul 24 20:16:49.479339 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.479274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx" event={"ID":"c90c70df-5a5c-43a2-9bdd-853a87651a08","Type":"ContainerStarted","Data":"3921d321c0b00ef084b1aa43ffd6b0f88c9cb368b11345650115ae2651bc61df"}
Jul 24 20:16:49.479339 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.479303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx" event={"ID":"c90c70df-5a5c-43a2-9bdd-853a87651a08","Type":"ContainerStarted","Data":"4ca114bf34ae1b7b6a34397ec7161da9741b7ed74920d09f27451b158d745bcc"}
Jul 24 20:16:49.482752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.482703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nxgsn" event={"ID":"502e4008-5e5a-4bee-8936-c5f799313c4f","Type":"ContainerStarted","Data":"c9f18c57f8f8b20f18625d7e4a86c66f4a49a733f57d67a91e5a64bd03a04bad"}
Jul 24 20:16:49.484736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.484695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8" event={"ID":"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5","Type":"ContainerStarted","Data":"dd1641996553c6d911592901d8f62af746d58bd8c02de205f658636f40105cb6"}
Jul 24 20:16:49.486215 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.486166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v" event={"ID":"00535512-56a8-4c9e-b62c-5b9e49dc692f","Type":"ContainerStarted","Data":"fee0a699fb01864a1aff6b0aeb33e965410937645c752c0eeb6b7c6c91831ddb"}
Jul 24 20:16:49.490641 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.490561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2" event={"ID":"5bd7b2fa-1edb-4163-8855-967b3ded1df6","Type":"ContainerStarted","Data":"476528bd35ebc9e2f8759c42da58a11a232a9fd5a375c09f177dc1dd994ab783"}
Jul 24 20:16:49.493694 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.493340    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gccc9" podStartSLOduration=2.16667998 podStartE2EDuration="20.493322689s" totalImagesPullingTime="18.326642709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:16:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.492133841 +0000 UTC m=+75.845710998" watchObservedRunningTime="2026-07-24 20:16:49.493322689 +0000 UTC m=+75.846899832"
Jul 24 20:16:49.495031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.495003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tlc7" event={"ID":"c249c9d6-0606-4b78-9ce7-281e837c2bee","Type":"ContainerStarted","Data":"929fd58ff5fc80c150bec781119d4a6773a3dcb63b6e2597e8b20dc4c7d54897"}
Jul 24 20:16:49.495142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.495054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tlc7" event={"ID":"c249c9d6-0606-4b78-9ce7-281e837c2bee","Type":"ContainerStarted","Data":"f72d7eec21ea77a14373e5a2cfddea4c26c0035a2a9f9efb13e640625e6f1063"}
Jul 24 20:16:49.497979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.497795    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405" exitCode=0
Jul 24 20:16:49.497979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.497873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"}
Jul 24 20:16:49.497979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.497929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"6bd971e2af3588a3229bebdc441ebde9185b41e0db37ae4f5e0dddc607ad3d33"}
Jul 24 20:16:49.501962 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.501936    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjmk" event={"ID":"86b45308-92d0-450f-bb8d-4ec2f4dd12cc","Type":"ContainerStarted","Data":"40b60cb31fd47640cd8a2c6e402a36acb7bfca4ce54f8cca1d4b8d7076ae2caf"}
Jul 24 20:16:49.502051 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.501987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwjmk" event={"ID":"86b45308-92d0-450f-bb8d-4ec2f4dd12cc","Type":"ContainerStarted","Data":"0bc49daad503c09fca9c24b19fef4bce773e51234a473e741a5cfc588d45889f"}
Jul 24 20:16:49.506155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.506128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"b30158fe63471e7a5cc01c61b8f8f92bd6447e7b5412cd8067d54a334645d106"}
Jul 24 20:16:49.506243 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.506198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"ad5333a7e444f113f98365bd845415b496953bdd4cdb87182f8b35c0b8e55776"}
Jul 24 20:16:49.506315 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.506241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"2fc3926fcc96f6fd8da80417dac8a6fc9a49563444186ee21c48a2de247da66b"}
Jul 24 20:16:49.508144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.508115    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="1b1680c2aaec0e18c46af2055a8752c690c9131c79899cc3b34dbf53cd210bdc" exitCode=0
Jul 24 20:16:49.508231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.508182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"1b1680c2aaec0e18c46af2055a8752c690c9131c79899cc3b34dbf53cd210bdc"}
Jul 24 20:16:49.516420 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.516371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-dvmg2" podStartSLOduration=3.163828528 podStartE2EDuration="10.516353176s" totalImagesPullingTime="7.352524648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.514522807 +0000 UTC m=+75.868099958" watchObservedRunningTime="2026-07-24 20:16:49.516353176 +0000 UTC m=+75.869930316"
Jul 24 20:16:49.521438 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.521408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-8pzq7" event={"ID":"9005214a-0920-4d14-a92b-c9a7e9b20617","Type":"ContainerStarted","Data":"fb10b59e6a6a4c72d945143c5dd707ad2da046b1f58ea2363c56c6a534f74e3c"}
Jul 24 20:16:49.535752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.534678    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nxgsn" podStartSLOduration=67.275064054 podStartE2EDuration="1m15.534658399s" totalImagesPullingTime="8.259594345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.534548682 +0000 UTC m=+75.888125836" watchObservedRunningTime="2026-07-24 20:16:49.534658399 +0000 UTC m=+75.888235540"
Jul 24 20:16:49.557058 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.555875    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m6v27" podStartSLOduration=67.316042598 podStartE2EDuration="1m15.555856683s" totalImagesPullingTime="8.239814085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.555470075 +0000 UTC m=+75.909047238" watchObservedRunningTime="2026-07-24 20:16:49.555856683 +0000 UTC m=+75.909433824"
Jul 24 20:16:49.575938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.575884    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-k9shx" podStartSLOduration=3.5573287479999998 podStartE2EDuration="10.575863981s" totalImagesPullingTime="7.018535233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.575462876 +0000 UTC m=+75.929040017" watchObservedRunningTime="2026-07-24 20:16:49.575863981 +0000 UTC m=+75.929441122"
Jul 24 20:16:49.648269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.647571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2tlc7" podStartSLOduration=68.633785799 podStartE2EDuration="1m15.647550647s" totalImagesPullingTime="7.013764848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.646662408 +0000 UTC m=+76.000239576" watchObservedRunningTime="2026-07-24 20:16:49.647550647 +0000 UTC m=+76.001127788"
Jul 24 20:16:49.671486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.670701    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dwjmk" podStartSLOduration=9.095496926 podStartE2EDuration="10.670680067s" totalImagesPullingTime="1.425145733s" totalInitContainerRuntime="150.037408ms" isStatefulPod=true podCreationTimestamp="2026-07-24 20:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:49.668280251 +0000 UTC m=+76.021857392" watchObservedRunningTime="2026-07-24 20:16:49.670680067 +0000 UTC m=+76.024257208"
Jul 24 20:16:49.760061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.759980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:49.836782 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:49.836707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:51.535091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.535004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"2302d8ec5afdc715303b4b3e6e235996ff7387cf920e1ac7ca6c1428c481de68"}
Jul 24 20:16:51.537208 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.537178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8" event={"ID":"947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5","Type":"ContainerStarted","Data":"0acbea10c4c5c3ffd3c703801aa7c931792b3ffc4f9b4aede010585014687876"}
Jul 24 20:16:51.539919 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.539880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v" event={"ID":"00535512-56a8-4c9e-b62c-5b9e49dc692f","Type":"ContainerStarted","Data":"d5837daf182a6c983a9b33e8e3d5b99800bc15fd59de1c87ba2dae74bfcfa156"}
Jul 24 20:16:51.540212 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.540162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:51.546832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.546809    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v"
Jul 24 20:16:51.558250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.556484    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8" podStartSLOduration=6.002669036 podStartE2EDuration="8.556465934s" totalImagesPullingTime="2.553796898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:51.555175442 +0000 UTC m=+77.908752580" watchObservedRunningTime="2026-07-24 20:16:51.556465934 +0000 UTC m=+77.910043076"
Jul 24 20:16:51.574224 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:51.574164    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-jk45v" podStartSLOduration=5.055206341 podStartE2EDuration="7.574142852s" totalImagesPullingTime="2.518936511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:51.572159778 +0000 UTC m=+77.925736940" watchObservedRunningTime="2026-07-24 20:16:51.574142852 +0000 UTC m=+77.927719992"
Jul 24 20:16:52.552351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.552111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"296051ddbedf122acde85e3416f9929336d131ace41299e3d3a30267a80ac6a7"}
Jul 24 20:16:52.552351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.552225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" event={"ID":"7eed69cd-9878-499b-a9b8-6dd5bd754c27","Type":"ContainerStarted","Data":"2396bb247d53d625b9f472168e3b863b00f47816037f823f0c776d9f950e0734"}
Jul 24 20:16:52.552351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.552307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:52.564196 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"32267d7438ba77b587e38d446fcdd64315c9610e071f72528bf4486658308c0d"}
Jul 24 20:16:52.564337 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"ee7f191b3e0f79c433ea163e1c1a6f4d86f515315b0bf88bc87f5ea554940c0b"}
Jul 24 20:16:52.564404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"ed46ae181b8e0a19244a127ffaf351f93f5fc3f571b164dbf15c2424a2e1ac4e"}
Jul 24 20:16:52.564459 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"5009100e1bfb4f2ef7cbba10e5531686b1a1efc6b84e4588021b87b72d05aaa5"}
Jul 24 20:16:52.564517 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"d781c0a389342614ece9cc141ac2d35ffd155295c82e71160e2568dfeef67387"}
Jul 24 20:16:52.564811 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.564781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerStarted","Data":"ed31227df19dbcd6df314b9be06de1fed90a1441d7fed42d3910c1551b478684"}
Jul 24 20:16:52.591221 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.591164    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9" podStartSLOduration=2.724579267 podStartE2EDuration="11.591142191s" totalImagesPullingTime="8.866562924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:52.58798008 +0000 UTC m=+78.941557225" watchObservedRunningTime="2026-07-24 20:16:52.591142191 +0000 UTC m=+78.944719332"
Jul 24 20:16:52.646844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:52.645324    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.218593264 podStartE2EDuration="12.645305042s" totalImagesPullingTime="9.420597914s" totalInitContainerRuntime="6.113864ms" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:52.643483533 +0000 UTC m=+78.997060702" watchObservedRunningTime="2026-07-24 20:16:52.645305042 +0000 UTC m=+78.998882185"
Jul 24 20:16:53.118327 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:53.118288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:53.171754 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:53.171720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:54.579995 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.579918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"}
Jul 24 20:16:54.579995 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.579980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"}
Jul 24 20:16:54.580476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.580011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"}
Jul 24 20:16:54.580476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.580034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"}
Jul 24 20:16:54.580476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.580055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"}
Jul 24 20:16:54.580476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.580077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerStarted","Data":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"}
Jul 24 20:16:54.615945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:54.615039    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.008526622 podStartE2EDuration="9.614012526s" totalImagesPullingTime="4.56975207s" totalInitContainerRuntime="35.733834ms" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:16:54.611787264 +0000 UTC m=+80.965364426" watchObservedRunningTime="2026-07-24 20:16:54.614012526 +0000 UTC m=+80.967589667"
Jul 24 20:16:55.939785 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:55.939752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:16:56.332202 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:56.332157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:56.434539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:56.434509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:56.568669 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:56.568613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:56.670096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:56.670018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:58.577023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:58.576988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b688554b-ghxn9"
Jul 24 20:16:59.312687 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:59.312657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:16:59.368980 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:16:59.368942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:03.316112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:03.316064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:03.416250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:03.416218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:03.793238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:03.793207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:03.852648 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:03.852615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:04.026907 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:04.026868    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:17:04.027075 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:04.026980    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:17:07.642961 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:07.642872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-8pzq7" event={"ID":"9005214a-0920-4d14-a92b-c9a7e9b20617","Type":"ContainerStarted","Data":"766d8be0ab6afa5c732e62899109845e81ae91c6657031217948cc41bf541531"}
Jul 24 20:17:07.643326 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:07.643053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:17:07.644446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:07.644420    2577 patch_prober.go:28] interesting pod/downloads-7c97dc4555-8pzq7 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.16:8080/\": dial tcp 10.134.0.16:8080: connect: connection refused" start-of-body=
Jul 24 20:17:07.644550 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:07.644469    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-8pzq7" podUID="9005214a-0920-4d14-a92b-c9a7e9b20617" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.16:8080/\": dial tcp 10.134.0.16:8080: connect: connection refused"
Jul 24 20:17:07.662786 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:07.662733    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-8pzq7" podStartSLOduration=4.143244829 podStartE2EDuration="22.662718559s" totalImagesPullingTime="18.51947373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:17:07.660624622 +0000 UTC m=+94.014201789" watchObservedRunningTime="2026-07-24 20:17:07.662718559 +0000 UTC m=+94.016295751"
Jul 24 20:17:08.506685 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:08.506642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:08.563100 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:08.563062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:08.664300 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:08.664272    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-8pzq7"
Jul 24 20:17:10.486706 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:10.486677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:10.549816 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:10.549782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:20.516101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:20.516066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:20.531818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:20.531794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m6v27"
Jul 24 20:17:20.615934 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:20.615901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:22.709130 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:22.709093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:22.766074 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:22.766042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:24.032305 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:24.032275    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:17:24.035957 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:24.035938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-588c96548f-5ffz8"
Jul 24 20:17:45.939792 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:45.939751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:17:46.009071 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:46.009048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:17:46.823784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:46.823757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:17:48.788220 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:48.788174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:48.860977 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:48.860939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:58.611188 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:58.611141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:59.259687 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.259636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:17:59.699403 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699318    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:17:59.699885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699752    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="alertmanager" containerID="cri-o://ed31227df19dbcd6df314b9be06de1fed90a1441d7fed42d3910c1551b478684" gracePeriod=120
Jul 24 20:17:59.699885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699820    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-metric" containerID="cri-o://ee7f191b3e0f79c433ea163e1c1a6f4d86f515315b0bf88bc87f5ea554940c0b" gracePeriod=120
Jul 24 20:17:59.700004 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699863    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="config-reloader" containerID="cri-o://d781c0a389342614ece9cc141ac2d35ffd155295c82e71160e2568dfeef67387" gracePeriod=120
Jul 24 20:17:59.700004 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699865    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-web" containerID="cri-o://5009100e1bfb4f2ef7cbba10e5531686b1a1efc6b84e4588021b87b72d05aaa5" gracePeriod=120
Jul 24 20:17:59.700004 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699907    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy" containerID="cri-o://ed46ae181b8e0a19244a127ffaf351f93f5fc3f571b164dbf15c2424a2e1ac4e" gracePeriod=120
Jul 24 20:17:59.700004 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.699879    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="prom-label-proxy" containerID="cri-o://32267d7438ba77b587e38d446fcdd64315c9610e071f72528bf4486658308c0d" gracePeriod=120
Jul 24 20:17:59.861556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861519    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="32267d7438ba77b587e38d446fcdd64315c9610e071f72528bf4486658308c0d" exitCode=0
Jul 24 20:17:59.861556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861541    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="ee7f191b3e0f79c433ea163e1c1a6f4d86f515315b0bf88bc87f5ea554940c0b" exitCode=0
Jul 24 20:17:59.861556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861547    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="ed46ae181b8e0a19244a127ffaf351f93f5fc3f571b164dbf15c2424a2e1ac4e" exitCode=0
Jul 24 20:17:59.861556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861552    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="d781c0a389342614ece9cc141ac2d35ffd155295c82e71160e2568dfeef67387" exitCode=0
Jul 24 20:17:59.861556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861558    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="ed31227df19dbcd6df314b9be06de1fed90a1441d7fed42d3910c1551b478684" exitCode=0
Jul 24 20:17:59.861818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"32267d7438ba77b587e38d446fcdd64315c9610e071f72528bf4486658308c0d"}
Jul 24 20:17:59.861818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"ee7f191b3e0f79c433ea163e1c1a6f4d86f515315b0bf88bc87f5ea554940c0b"}
Jul 24 20:17:59.861818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"ed46ae181b8e0a19244a127ffaf351f93f5fc3f571b164dbf15c2424a2e1ac4e"}
Jul 24 20:17:59.861818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"d781c0a389342614ece9cc141ac2d35ffd155295c82e71160e2568dfeef67387"}
Jul 24 20:17:59.861818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:17:59.861740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"ed31227df19dbcd6df314b9be06de1fed90a1441d7fed42d3910c1551b478684"}
Jul 24 20:18:00.870932 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:00.870901    2577 generic.go:356] "Generic (PLEG): container finished" podID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerID="5009100e1bfb4f2ef7cbba10e5531686b1a1efc6b84e4588021b87b72d05aaa5" exitCode=0
Jul 24 20:18:00.871223 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:00.870944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"5009100e1bfb4f2ef7cbba10e5531686b1a1efc6b84e4588021b87b72d05aaa5"}
Jul 24 20:18:00.947097 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:00.947070    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:01.018872 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.018847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.018901    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.018928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.018954    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.018986    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019040    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019072    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019147    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc59s\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") pod \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\" (UID: \"3934fd0f-8bfd-4ade-b4bd-f810e282f12b\") "
Jul 24 20:18:01.019348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:18:01.019708 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019515    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-main-db\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.020059 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.019907    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:01.020273 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.020249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:01.022448 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.022401    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out" (OuterVolumeSpecName: "config-out") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:18:01.023351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.023327    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:18:01.023465 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.023443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.023692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.023652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.023768 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.023732    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume" (OuterVolumeSpecName: "config-volume") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.023910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.023891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.024110 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.024082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.024618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.024572    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s" (OuterVolumeSpecName: "kube-api-access-nc59s") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "kube-api-access-nc59s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:18:01.028695 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.028671    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.035483 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.035457    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config" (OuterVolumeSpecName: "web-config") pod "3934fd0f-8bfd-4ade-b4bd-f810e282f12b" (UID: "3934fd0f-8bfd-4ade-b4bd-f810e282f12b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:01.119884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119851    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.119884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119877    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-main-tls\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.119884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119888    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-cluster-tls-config\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.119884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119897    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-web-config\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119908    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119916    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-config-out\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119924    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119932    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-metrics-client-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119940    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119949    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119958    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-tls-assets\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.120137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.119967    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc59s\" (UniqueName: \"kubernetes.io/projected/3934fd0f-8bfd-4ade-b4bd-f810e282f12b-kube-api-access-nc59s\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:01.878154 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.878126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3934fd0f-8bfd-4ade-b4bd-f810e282f12b","Type":"ContainerDied","Data":"e63104b8d77c6b55acfdcf6383ddc43ab60a45e55ffe7ea0ded415305684356c"}
Jul 24 20:18:01.878154 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.878150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:01.878643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.878209    2577 scope.go:117] "RemoveContainer" containerID="32267d7438ba77b587e38d446fcdd64315c9610e071f72528bf4486658308c0d"
Jul 24 20:18:01.887947 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.887926    2577 scope.go:117] "RemoveContainer" containerID="ee7f191b3e0f79c433ea163e1c1a6f4d86f515315b0bf88bc87f5ea554940c0b"
Jul 24 20:18:01.895039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.895022    2577 scope.go:117] "RemoveContainer" containerID="ed46ae181b8e0a19244a127ffaf351f93f5fc3f571b164dbf15c2424a2e1ac4e"
Jul 24 20:18:01.901622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.901584    2577 scope.go:117] "RemoveContainer" containerID="5009100e1bfb4f2ef7cbba10e5531686b1a1efc6b84e4588021b87b72d05aaa5"
Jul 24 20:18:01.905296 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.905271    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:18:01.909579 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.909559    2577 scope.go:117] "RemoveContainer" containerID="d781c0a389342614ece9cc141ac2d35ffd155295c82e71160e2568dfeef67387"
Jul 24 20:18:01.909718 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.909586    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:18:01.916426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.916410    2577 scope.go:117] "RemoveContainer" containerID="ed31227df19dbcd6df314b9be06de1fed90a1441d7fed42d3910c1551b478684"
Jul 24 20:18:01.923325 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.923307    2577 scope.go:117] "RemoveContainer" containerID="1b1680c2aaec0e18c46af2055a8752c690c9131c79899cc3b34dbf53cd210bdc"
Jul 24 20:18:01.938264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938236    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:18:01.938576 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938560    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="alertmanager"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938577    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="alertmanager"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938588    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="prom-label-proxy"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938614    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="prom-label-proxy"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938625    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="config-reloader"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938633    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="config-reloader"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938642    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-metric"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938651    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-metric"
Jul 24 20:18:01.938673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938669    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-web"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938677    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-web"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938692    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="init-config-reloader"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938701    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="init-config-reloader"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938721    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938730    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938810    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-metric"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938822    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="config-reloader"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938833    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938843    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="alertmanager"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938850    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="kube-rbac-proxy-web"
Jul 24 20:18:01.939047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.938860    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" containerName="prom-label-proxy"
Jul 24 20:18:01.942420 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.942401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:01.945279 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 20:18:01.945432 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945289    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 20:18:01.945576 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 20:18:01.945704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945586    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f9j4l\""
Jul 24 20:18:01.945704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945638    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 20:18:01.945704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945639    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 20:18:01.945898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945808    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 20:18:01.945898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 20:18:01.945898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.945891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 20:18:01.955234 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.955212    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 20:18:01.956957 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:01.956938    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:18:02.027412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-web-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpb5l\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-kube-api-access-fpb5l\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027615    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-out\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.027784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.027729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.128884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.128835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.128884 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.128890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129105 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129160 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129160 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-out\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-web-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpb5l\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-kube-api-access-fpb5l\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.129580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.129561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.130946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.130896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d963d8f-1885-4f03-b67d-d13cff6dc43c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.132362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.132317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-volume\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.132487 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.132391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.132883 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.132855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.132883 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.132862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-web-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.133209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.133183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8d963d8f-1885-4f03-b67d-d13cff6dc43c-config-out\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.133315 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.133294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.133396 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.133375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.133432 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.133417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.133981 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.133962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8d963d8f-1885-4f03-b67d-d13cff6dc43c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.138912 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.138894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpb5l\" (UniqueName: \"kubernetes.io/projected/8d963d8f-1885-4f03-b67d-d13cff6dc43c-kube-api-access-fpb5l\") pod \"alertmanager-main-0\" (UID: \"8d963d8f-1885-4f03-b67d-d13cff6dc43c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.151903 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.151871    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3934fd0f-8bfd-4ade-b4bd-f810e282f12b" path="/var/lib/kubelet/pods/3934fd0f-8bfd-4ade-b4bd-f810e282f12b/volumes"
Jul 24 20:18:02.253026 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.252999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 20:18:02.390448 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.390420    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 20:18:02.393346 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:18:02.393321    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d963d8f_1885_4f03_b67d_d13cff6dc43c.slice/crio-d718f34726b183896e4c6aa8d30c1a97439be487f19a66a37c8d0732ea72d8e7 WatchSource:0}: Error finding container d718f34726b183896e4c6aa8d30c1a97439be487f19a66a37c8d0732ea72d8e7: Status 404 returned error can't find the container with id d718f34726b183896e4c6aa8d30c1a97439be487f19a66a37c8d0732ea72d8e7
Jul 24 20:18:02.883411 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.883377    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d963d8f-1885-4f03-b67d-d13cff6dc43c" containerID="41ea33e2c3d3bcf35cd87290c9933fc370a68b5ed7b18c0267f2301ba185a78a" exitCode=0
Jul 24 20:18:02.883842 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.883469    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerDied","Data":"41ea33e2c3d3bcf35cd87290c9933fc370a68b5ed7b18c0267f2301ba185a78a"}
Jul 24 20:18:02.883842 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:02.883551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"d718f34726b183896e4c6aa8d30c1a97439be487f19a66a37c8d0732ea72d8e7"}
Jul 24 20:18:03.893644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"a58f0bfe463b7f14de5464a69c4e69d028a18bb2d08e9141df845eaeed1f4ccf"}
Jul 24 20:18:03.894023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"5247877e0222cea3acb4d47ee34273a60c559294177fc9e4349a06ea37cdfeb0"}
Jul 24 20:18:03.894023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"bfc8c4f7a61611bccbb54eed1fdaff7eaa440a8eacbbaef256519ff26e640adc"}
Jul 24 20:18:03.894023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"2f859229a581728e5c5a03e0c87e3f9bc30bfb993adb2ade68895a5eb5f0ddba"}
Jul 24 20:18:03.894023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"f0a2b8996a20f0132947da7593314ab97435e8954c73207271f74ec6b1937caf"}
Jul 24 20:18:03.894023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.893773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8d963d8f-1885-4f03-b67d-d13cff6dc43c","Type":"ContainerStarted","Data":"d3205eb182c6d5caacaaa06876548df7baf238718fb09d36a070b032e81d18ee"}
Jul 24 20:18:03.943484 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:03.943432    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.929653014 podStartE2EDuration="2.943413544s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.76053ms" isStatefulPod=false podCreationTimestamp="2026-07-24 20:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:18:03.941189399 +0000 UTC m=+150.294766561" watchObservedRunningTime="2026-07-24 20:18:03.943413544 +0000 UTC m=+150.296990683"
Jul 24 20:18:04.014216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.014178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:04.016096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.014636    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8" gracePeriod=600
Jul 24 20:18:04.016096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.015160    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-web" containerID="cri-o://a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123" gracePeriod=600
Jul 24 20:18:04.016096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.015287    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy" containerID="cri-o://e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49" gracePeriod=600
Jul 24 20:18:04.016096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.015391    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="thanos-sidecar" containerID="cri-o://1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c" gracePeriod=600
Jul 24 20:18:04.016096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.015473    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="config-reloader" containerID="cri-o://df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91" gracePeriod=600
Jul 24 20:18:04.016865 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.016141    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="prometheus" containerID="cri-o://101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81" gracePeriod=600
Jul 24 20:18:04.266686 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.266650    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:04.350520 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350559    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350638    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350667    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350694    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350849    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350875    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.350945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cftwk\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.350989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351076    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351101    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls\") pod \"60ce60cd-7b93-441b-864d-66002c82126c\" (UID: \"60ce60cd-7b93-441b-864d-66002c82126c\") "
Jul 24 20:18:04.351333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351319    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.351859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351547    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.351859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351567    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.351859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351582    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.351859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351798    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.352077 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.351617    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-configmap-metrics-client-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.352846 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.352566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.353357 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.353327    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:18:04.353812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.353787    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:18:04.354195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.354167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.354419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.354372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.355685 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.355651    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out" (OuterVolumeSpecName: "config-out") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:18:04.355806 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.355765    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:18:04.356047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config" (OuterVolumeSpecName: "config") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.356143 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.356226 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356205    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.356405 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.356493 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356448    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk" (OuterVolumeSpecName: "kube-api-access-cftwk") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "kube-api-access-cftwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:18:04.356864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.356942 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.356891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.367012 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.366985    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config" (OuterVolumeSpecName: "web-config") pod "60ce60cd-7b93-441b-864d-66002c82126c" (UID: "60ce60cd-7b93-441b-864d-66002c82126c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453049    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453076    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-grpc-tls\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453087    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453096    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-web-config\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453106    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453115    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453123    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-tls-assets\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453131    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-metrics-client-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453139    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453147    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453156    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-secret-kube-rbac-proxy\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453166    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cftwk\" (UniqueName: \"kubernetes.io/projected/60ce60cd-7b93-441b-864d-66002c82126c-kube-api-access-cftwk\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453174    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-config-out\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453181    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-db\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453189    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/60ce60cd-7b93-441b-864d-66002c82126c-config\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.453415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.453197    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60ce60cd-7b93-441b-864d-66002c82126c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:18:04.901385 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901349    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8" exitCode=0
Jul 24 20:18:04.901385 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901375    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49" exitCode=0
Jul 24 20:18:04.901385 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901387    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123" exitCode=0
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901398    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c" exitCode=0
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901405    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91" exitCode=0
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901412    2577 generic.go:356] "Generic (PLEG): container finished" podID="60ce60cd-7b93-441b-864d-66002c82126c" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81" exitCode=0
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901418    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901566    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"}
Jul 24 20:18:04.901909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.901644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60ce60cd-7b93-441b-864d-66002c82126c","Type":"ContainerDied","Data":"6bd971e2af3588a3229bebdc441ebde9185b41e0db37ae4f5e0dddc607ad3d33"}
Jul 24 20:18:04.909794 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.909780    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.916682 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.916661    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.923379 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.923360    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.925466 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.925445    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:04.930153 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.930135    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:04.930814 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.930799    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.937399 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.937383    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.944360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.944338    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.950911 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.950897    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.951154 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.951137    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.951192 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951163    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.951232 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951194    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.951433 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.951410    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.951518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951438    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.951518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951455    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.951707 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.951691    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.951751 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951713    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.951751 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951727    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.951924 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.951908    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.951964 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.951964 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.951943    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.952343 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.952284    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.952515 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.952349    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.952515 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.952368    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.952811 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.952785    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.952960 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.952818    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.952960 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.952839    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.953198 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:18:04.953178    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.953272 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953205    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.953272 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953224    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.953454 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953433    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.953518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953455    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.953672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953654    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.953760 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953673    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.953885 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953866    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.953943 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.953887    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.954109 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954090    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.954109 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954107    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.954287 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954270    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.954338 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954289    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.954499 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954478    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.954499 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954498    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.954741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954723    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.954808 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954741    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.954808 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954767    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:04.954982 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954964    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.955034 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.954981    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.955145 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955130    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="thanos-sidecar"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955151    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="thanos-sidecar"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955165    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955174    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955174    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955186    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955189    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="init-config-reloader"
Jul 24 20:18:04.955203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955198    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="init-config-reloader"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955211    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-thanos"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955220    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-thanos"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955231    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="config-reloader"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955239    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="config-reloader"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955255    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="prometheus"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955263    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="prometheus"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955275    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-web"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955283    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-web"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955350    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="config-reloader"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955357    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955381    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955362    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-thanos"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955425    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="thanos-sidecar"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955431    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy-web"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955441    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="prometheus"
Jul 24 20:18:04.955500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955447    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ce60cd-7b93-441b-864d-66002c82126c" containerName="kube-rbac-proxy"
Jul 24 20:18:04.956085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955650    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.956085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955671    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.956085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955896    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.956085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.955916    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.956231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956106    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.956231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956119    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.956348 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956327    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.956404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956352    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.956783 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956763    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.956783 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.956783    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.957130 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957102    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.957130 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957128    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.957411 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957390    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.957411 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957409    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.957662 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957635    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.957791 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957667    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.957958 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957931    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.958042 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.957961    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.958261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958236    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.958349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958262    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.958536 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958502    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.958618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958553    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.958818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958797    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.958818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.958817    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.959061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959044    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.959115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959063    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.959256 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959237    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.959332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959258    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.959490 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959473    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.959563 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959492    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.959733 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959714    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.959811 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959736    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.959931 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959919    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:04.959976 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959926    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.959976 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.959941    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.960394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960367    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.960394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960389    2577 scope.go:117] "RemoveContainer" containerID="5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"
Jul 24 20:18:04.960650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960631    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8"} err="failed to get container status \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": rpc error: code = NotFound desc = could not find container \"5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8\": container with ID starting with 5ab2054f4e474cb63acca6a67c9d0d5b4b61f8da4cf9bf6511921a1d9a6c94a8 not found: ID does not exist"
Jul 24 20:18:04.960650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960650    2577 scope.go:117] "RemoveContainer" containerID="e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"
Jul 24 20:18:04.960894 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960877    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49"} err="failed to get container status \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": rpc error: code = NotFound desc = could not find container \"e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49\": container with ID starting with e4a4ccd47827308915e4c32d0f0428ebfebddee12e48f75211d9502a98821c49 not found: ID does not exist"
Jul 24 20:18:04.960957 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.960893    2577 scope.go:117] "RemoveContainer" containerID="a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"
Jul 24 20:18:04.961086 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961068    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123"} err="failed to get container status \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": rpc error: code = NotFound desc = could not find container \"a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123\": container with ID starting with a935882142ad8fbb60a88727958b6f73145573632944a4c7581661470101b123 not found: ID does not exist"
Jul 24 20:18:04.961131 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961087    2577 scope.go:117] "RemoveContainer" containerID="1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"
Jul 24 20:18:04.961310 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961294    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c"} err="failed to get container status \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": rpc error: code = NotFound desc = could not find container \"1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c\": container with ID starting with 1eb432ca6c59463c611b8b5c348b9c51f7e5c5175bf195f93a3bcd8cb2c3df8c not found: ID does not exist"
Jul 24 20:18:04.961359 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961319    2577 scope.go:117] "RemoveContainer" containerID="df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"
Jul 24 20:18:04.961553 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961512    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91"} err="failed to get container status \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": rpc error: code = NotFound desc = could not find container \"df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91\": container with ID starting with df2208afc1fd4dd837f6c0143b02904e98f9f379ca53efafa2ecd94b2eceed91 not found: ID does not exist"
Jul 24 20:18:04.961553 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961534    2577 scope.go:117] "RemoveContainer" containerID="101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"
Jul 24 20:18:04.961800 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961783    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81"} err="failed to get container status \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": rpc error: code = NotFound desc = could not find container \"101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81\": container with ID starting with 101cb5d68d18c87ed22a2634d7e0ecd6c2af8600957a58c7289c4944ef23db81 not found: ID does not exist"
Jul 24 20:18:04.961860 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.961800    2577 scope.go:117] "RemoveContainer" containerID="24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"
Jul 24 20:18:04.962058 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962040    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405"} err="failed to get container status \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": rpc error: code = NotFound desc = could not find container \"24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405\": container with ID starting with 24ac9c5747ee7858213a4b43d415f555eea326f520e8815e25487b489f644405 not found: ID does not exist"
Jul 24 20:18:04.962627 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962584    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m7vxs\""
Jul 24 20:18:04.962745 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962703    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 20:18:04.962900 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 20:18:04.962900 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8mab07kva0fbb\""
Jul 24 20:18:04.963073 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.962942    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 20:18:04.963073 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963021    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 20:18:04.963073 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963051    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 20:18:04.963264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963123    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 20:18:04.963264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 20:18:04.963264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963161    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 20:18:04.963264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963257    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 20:18:04.963473 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963373    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 20:18:04.963473 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963451    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 20:18:04.963637 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.963623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 20:18:04.974018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.969648    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 20:18:04.975613 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.975572    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 20:18:04.976039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:04.976023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:05.056887 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.056805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.056887 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.056838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.056895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.056916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.056940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057584 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057584 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057584 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057584 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv667\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-kube-api-access-zv667\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.057763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.057698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv667\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-kube-api-access-zv667\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158692    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.158985 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.158978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159712 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159772 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.159963 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.159620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.160644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.160538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.160644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.160625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.160856 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.160831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.161731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.161668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.162643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.162621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.162832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.162697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.162993 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.162797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.163155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.163135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.163482 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.163458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.164142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.164118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.164231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.164168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.164832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.164808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.165024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.165007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-config\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.165167 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.165146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.165232 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.165204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.165341 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.165324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.167967 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.167947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv667\" (UniqueName: \"kubernetes.io/projected/3508d467-ba7a-4b4d-ab1f-5917e8d946b7-kube-api-access-zv667\") pod \"prometheus-k8s-0\" (UID: \"3508d467-ba7a-4b4d-ab1f-5917e8d946b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.276435 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.276401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:05.414005 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.413823    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 20:18:05.415933 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:18:05.415910    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3508d467_ba7a_4b4d_ab1f_5917e8d946b7.slice/crio-c8d14d6c2dd8f11d556b6b74addd5665883b85fc20e94fd1e58741360509cda4 WatchSource:0}: Error finding container c8d14d6c2dd8f11d556b6b74addd5665883b85fc20e94fd1e58741360509cda4: Status 404 returned error can't find the container with id c8d14d6c2dd8f11d556b6b74addd5665883b85fc20e94fd1e58741360509cda4
Jul 24 20:18:05.909698 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.909658    2577 generic.go:356] "Generic (PLEG): container finished" podID="3508d467-ba7a-4b4d-ab1f-5917e8d946b7" containerID="0d206ec1f39be6a40211cdc0eabf0afcb2f9b73d6c548ef949a1e791c7de0c62" exitCode=0
Jul 24 20:18:05.910129 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.909715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerDied","Data":"0d206ec1f39be6a40211cdc0eabf0afcb2f9b73d6c548ef949a1e791c7de0c62"}
Jul 24 20:18:05.910129 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:05.909773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"c8d14d6c2dd8f11d556b6b74addd5665883b85fc20e94fd1e58741360509cda4"}
Jul 24 20:18:06.156736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.156151    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ce60cd-7b93-441b-864d-66002c82126c" path="/var/lib/kubelet/pods/60ce60cd-7b93-441b-864d-66002c82126c/volumes"
Jul 24 20:18:06.917033 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.916996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"ec83f5d5dabf4abe0d1b641fc0d4258da65c6d1dd0683dfc55f0a45276d3d9e6"}
Jul 24 20:18:06.917450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.917117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"ce68c5116b3289b1c01ce886ee4060d2a58d5120bd558241ad05846913a6456f"}
Jul 24 20:18:06.917450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.917167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"b4b0b0f6cccb86ae3b1cd853030557de06a6ba6ec8e0a1e3789f5ff6b968a6e5"}
Jul 24 20:18:06.917450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.917207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"c6193967cf9c3fc32d3c14ef8543c7ded9687c2dd954620ede41ab6965e8e034"}
Jul 24 20:18:06.917450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.917246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"99653fde165afb02461c5d91b972ceeab7b52eecc387f69ccc3b6d0d651829cb"}
Jul 24 20:18:06.917450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.917283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3508d467-ba7a-4b4d-ab1f-5917e8d946b7","Type":"ContainerStarted","Data":"6bc4211dfd38837baa26e9136dc06eb241da6f915c6e6130bb1c856e5d2261c4"}
Jul 24 20:18:06.947124 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:06.947070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9286318810000003 podStartE2EDuration="2.947053766s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.421885ms" isStatefulPod=false podCreationTimestamp="2026-07-24 20:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:18:06.945126337 +0000 UTC m=+153.298703516" watchObservedRunningTime="2026-07-24 20:18:06.947053766 +0000 UTC m=+153.300630904"
Jul 24 20:18:08.387826 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:08.387787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:08.447558 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:08.447531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:10.276759 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:10.276729    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:18:16.267410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:16.267375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:18.861076 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:18.861047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:18.914977 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:18.914952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:18.967538 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:18.967504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:57.029530 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:57.029500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:18:57.084186 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:18:57.084156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:05.277281 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:05.277246    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:19:05.293554 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:05.293525    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:19:06.164965 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:06.164938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 20:19:13.892940 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:13.892903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:13.948756 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:13.948724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:14.386312 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:14.386274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:14.484412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:14.484350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:14.582552 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:14.582517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:15.123544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:15.123511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:25.118540 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:25.118509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:25.239829 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:25.239789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:25.293173 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:25.293143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:25.762206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:25.762173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:25.863236 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:25.863202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:33.797102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:33.797066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:33.897218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:33.897187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:33.960373 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:33.960340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:19:41.737346 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:19:41.737314    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:01.086009 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:01.085976    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:01.185663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:01.185635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:27.808354 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:27.808320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:29.047115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:29.047082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:31.409664 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:31.409623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:31.508253 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:31.508223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:32.725915 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:32.725886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:32.823698 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:32.823657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:33.035105 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:33.035075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:33.218365 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:33.218338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:34.008574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:34.008546    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 20:20:35.564227 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:35.564195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:35.783242 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:35.783209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:36.321949 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.321917    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"]
Jul 24 20:20:36.324206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.324191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.326767 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.326745    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 20:20:36.326880 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.326745    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 20:20:36.327748 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.327732    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mrm55\""
Jul 24 20:20:36.333864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.333845    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"]
Jul 24 20:20:36.339861 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.339842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzh4\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-kube-api-access-cqzh4\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.339951 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.339876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.440702 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.440670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzh4\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-kube-api-access-cqzh4\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.440845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.440712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.449734 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.449704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.449867 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.449848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzh4\" (UniqueName: \"kubernetes.io/projected/7d6fefff-e762-4f85-b79a-5878de260fe8-kube-api-access-cqzh4\") pod \"cert-manager-cainjector-559cdd8c87-dp6xn\" (UID: \"7d6fefff-e762-4f85-b79a-5878de260fe8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.646381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.646292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"
Jul 24 20:20:36.774793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.774763    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn"]
Jul 24 20:20:36.777703 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:20:36.777674    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d6fefff_e762_4f85_b79a_5878de260fe8.slice/crio-90906472dad80f28fee3051a5659ff20d79a3b8c277c4548241079c4c3abf6e5 WatchSource:0}: Error finding container 90906472dad80f28fee3051a5659ff20d79a3b8c277c4548241079c4c3abf6e5: Status 404 returned error can't find the container with id 90906472dad80f28fee3051a5659ff20d79a3b8c277c4548241079c4c3abf6e5
Jul 24 20:20:36.779497 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:36.779480    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:20:37.485308 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:37.485272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn" event={"ID":"7d6fefff-e762-4f85-b79a-5878de260fe8","Type":"ContainerStarted","Data":"90906472dad80f28fee3051a5659ff20d79a3b8c277c4548241079c4c3abf6e5"}
Jul 24 20:20:38.184468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:38.184427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:38.242667 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:38.242631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:40.501305 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:40.501216    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn" event={"ID":"7d6fefff-e762-4f85-b79a-5878de260fe8","Type":"ContainerStarted","Data":"a7a3ce44d23b7ca0cf67f1bf0d829f14d837e56ad41fe306f757995217422069"}
Jul 24 20:20:40.517388 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:40.517342    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dp6xn" podStartSLOduration=1.286576818 podStartE2EDuration="4.517327374s" totalImagesPullingTime="3.230750556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:20:40.516334266 +0000 UTC m=+306.869911406" watchObservedRunningTime="2026-07-24 20:20:40.517327374 +0000 UTC m=+306.870904514"
Jul 24 20:20:42.807835 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:42.807801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:44.000929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:44.000896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:44.101512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:44.101481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:45.529064 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.529031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp7qq"]
Jul 24 20:20:45.532442 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.532425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.535699 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.535676    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w2z7p\""
Jul 24 20:20:45.542840 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.542820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp7qq"]
Jul 24 20:20:45.602369 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.602342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:45.621446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.621424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg255\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-kube-api-access-sg255\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.621526 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.621482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.659840 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.659809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:45.722842 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.722814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.722962 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.722851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg255\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-kube-api-access-sg255\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.732059 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.732034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.732332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.732312    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg255\" (UniqueName: \"kubernetes.io/projected/8ab16f1c-543d-492e-8ef4-f85f44fa71e6-kube-api-access-sg255\") pod \"cert-manager-55b4d954d5-sp7qq\" (UID: \"8ab16f1c-543d-492e-8ef4-f85f44fa71e6\") " pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.841297 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.841216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sp7qq"
Jul 24 20:20:45.975326 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:45.975207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp7qq"]
Jul 24 20:20:45.977845 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:20:45.977819    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ab16f1c_543d_492e_8ef4_f85f44fa71e6.slice/crio-3985342f2f377956b6c56f347fa91af609c8ee04088f7172c0134f8e8323ff96 WatchSource:0}: Error finding container 3985342f2f377956b6c56f347fa91af609c8ee04088f7172c0134f8e8323ff96: Status 404 returned error can't find the container with id 3985342f2f377956b6c56f347fa91af609c8ee04088f7172c0134f8e8323ff96
Jul 24 20:20:46.526777 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:46.526739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sp7qq" event={"ID":"8ab16f1c-543d-492e-8ef4-f85f44fa71e6","Type":"ContainerStarted","Data":"e0ce84415ea2c16d44150a95f314c9ba46b96fa51255eac55af8f1f6ecdb4498"}
Jul 24 20:20:46.526948 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:46.526789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sp7qq" event={"ID":"8ab16f1c-543d-492e-8ef4-f85f44fa71e6","Type":"ContainerStarted","Data":"3985342f2f377956b6c56f347fa91af609c8ee04088f7172c0134f8e8323ff96"}
Jul 24 20:20:46.541744 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:46.541690    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sp7qq" podStartSLOduration=1.541670201 podStartE2EDuration="1.541670201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:20:46.541640486 +0000 UTC m=+312.895217626" watchObservedRunningTime="2026-07-24 20:20:46.541670201 +0000 UTC m=+312.895247342"
Jul 24 20:20:51.065860 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:51.065822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:51.166481 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:51.166452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:20:53.015572 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:20:53.015539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:09.317144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.317107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:09.408099 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.408070    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"]
Jul 24 20:21:09.411993 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.411977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.416236 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416213    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 20:21:09.416236 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416230    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 20:21:09.416672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416637    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z95z7\""
Jul 24 20:21:09.416672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416665    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 20:21:09.416828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416687    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 20:21:09.416828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.416690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 20:21:09.429259 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.429230    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"]
Jul 24 20:21:09.463580 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.463550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:09.522404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.522371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-metrics-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.522404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.522409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de498202-8366-4eff-bcc1-3defe8c51a94-manager-config\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.522667 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.522434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhprj\" (UniqueName: \"kubernetes.io/projected/de498202-8366-4eff-bcc1-3defe8c51a94-kube-api-access-nhprj\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.522667 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.522505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.545611 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.545566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:09.623533 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.623448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-metrics-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.623533 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.623490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de498202-8366-4eff-bcc1-3defe8c51a94-manager-config\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.623533 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.623524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhprj\" (UniqueName: \"kubernetes.io/projected/de498202-8366-4eff-bcc1-3defe8c51a94-kube-api-access-nhprj\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.623823 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.623632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.624196 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.624178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/de498202-8366-4eff-bcc1-3defe8c51a94-manager-config\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.626049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.625992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.626188 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.626055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/de498202-8366-4eff-bcc1-3defe8c51a94-metrics-cert\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.640322 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.640298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhprj\" (UniqueName: \"kubernetes.io/projected/de498202-8366-4eff-bcc1-3defe8c51a94-kube-api-access-nhprj\") pod \"lws-controller-manager-779f8ddb8c-qbk9s\" (UID: \"de498202-8366-4eff-bcc1-3defe8c51a94\") " pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.722374 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.722341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:09.878708 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:09.878681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"]
Jul 24 20:21:09.880528 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:21:09.880503    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde498202_8366_4eff_bcc1_3defe8c51a94.slice/crio-180ece4c28a701f9ef23012347843ef7fcf410405abb326404e782818e775945 WatchSource:0}: Error finding container 180ece4c28a701f9ef23012347843ef7fcf410405abb326404e782818e775945: Status 404 returned error can't find the container with id 180ece4c28a701f9ef23012347843ef7fcf410405abb326404e782818e775945
Jul 24 20:21:10.022034 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.021996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.183310 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.183217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.322992 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.322961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.357561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.357529    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"]
Jul 24 20:21:10.360664 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.360645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.363570 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.363545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 20:21:10.363710 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.363606    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 20:21:10.364008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.363990    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 20:21:10.364008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.363989    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lmv5f\""
Jul 24 20:21:10.364185 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.364117    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 20:21:10.382079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.382054    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"]
Jul 24 20:21:10.430392 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.430362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9zk\" (UniqueName: \"kubernetes.io/projected/08a77acb-41f9-406f-8607-b4f85abfb5c1-kube-api-access-fn9zk\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.430547 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.430415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.430547 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.430441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-tmp\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.430665 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.430549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.430665 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.430629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-manifests\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.515990 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.515919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.531797 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.531775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.531936 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.531805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-manifests\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.531936 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.531833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9zk\" (UniqueName: \"kubernetes.io/projected/08a77acb-41f9-406f-8607-b4f85abfb5c1-kube-api-access-fn9zk\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.531936 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.531878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.531936 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.531904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-tmp\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.532208 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.532186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-manifests\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.534274 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.534252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a77acb-41f9-406f-8607-b4f85abfb5c1-tmp\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.534420 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.534362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.534480 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.534464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08a77acb-41f9-406f-8607-b4f85abfb5c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.544897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.544880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9zk\" (UniqueName: \"kubernetes.io/projected/08a77acb-41f9-406f-8607-b4f85abfb5c1-kube-api-access-fn9zk\") pod \"opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl\" (UID: \"08a77acb-41f9-406f-8607-b4f85abfb5c1\") " pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.616082 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.616057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.619280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.619252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s" event={"ID":"de498202-8366-4eff-bcc1-3defe8c51a94","Type":"ContainerStarted","Data":"180ece4c28a701f9ef23012347843ef7fcf410405abb326404e782818e775945"}
Jul 24 20:21:10.673442 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.673417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:10.726619 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.726526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.821929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.821761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:10.826057 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.826030    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"]
Jul 24 20:21:10.830226 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:21:10.830193    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a77acb_41f9_406f_8607_b4f85abfb5c1.slice/crio-20cd898449ec414cac09cb54f2a7bbff9d814315586d3dbee2da59f001ec6c3e WatchSource:0}: Error finding container 20cd898449ec414cac09cb54f2a7bbff9d814315586d3dbee2da59f001ec6c3e: Status 404 returned error can't find the container with id 20cd898449ec414cac09cb54f2a7bbff9d814315586d3dbee2da59f001ec6c3e
Jul 24 20:21:10.918410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:10.918378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:11.627426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:11.627355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl" event={"ID":"08a77acb-41f9-406f-8607-b4f85abfb5c1","Type":"ContainerStarted","Data":"20cd898449ec414cac09cb54f2a7bbff9d814315586d3dbee2da59f001ec6c3e"}
Jul 24 20:21:14.643635 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:14.643522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s" event={"ID":"de498202-8366-4eff-bcc1-3defe8c51a94","Type":"ContainerStarted","Data":"f564309fa145c1ae5b0567a1cffc669685e5d04ef6630dadb747f24e9f1b7ce5"}
Jul 24 20:21:14.644142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:14.643640    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:14.645233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:14.645208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl" event={"ID":"08a77acb-41f9-406f-8607-b4f85abfb5c1","Type":"ContainerStarted","Data":"362577bc659030f6e005ce3a3910ecd8f0a5c8b25c5351f24d9b6b640291e988"}
Jul 24 20:21:14.662391 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:14.662351    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s" podStartSLOduration=1.15741863 podStartE2EDuration="5.662338696s" totalImagesPullingTime="4.504920066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:21:14.66101191 +0000 UTC m=+341.014589075" watchObservedRunningTime="2026-07-24 20:21:14.662338696 +0000 UTC m=+341.015915835"
Jul 24 20:21:15.650943 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:15.650907    2577 generic.go:356] "Generic (PLEG): container finished" podID="08a77acb-41f9-406f-8607-b4f85abfb5c1" containerID="362577bc659030f6e005ce3a3910ecd8f0a5c8b25c5351f24d9b6b640291e988" exitCode=0
Jul 24 20:21:15.651360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:15.651002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl" event={"ID":"08a77acb-41f9-406f-8607-b4f85abfb5c1","Type":"ContainerDied","Data":"362577bc659030f6e005ce3a3910ecd8f0a5c8b25c5351f24d9b6b640291e988"}
Jul 24 20:21:16.658644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:16.658573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl" event={"ID":"08a77acb-41f9-406f-8607-b4f85abfb5c1","Type":"ContainerStarted","Data":"feac15162ff551d4915148fcc2a8b693b999d366caba3d6f66f0f9e6aa11acab"}
Jul 24 20:21:16.659029 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:16.658679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:16.682302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:16.682252    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl" podStartSLOduration=1.731489203 podStartE2EDuration="6.682237929s" totalImagesPullingTime="3.89859752s" totalInitContainerRuntime="1.052151206s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:21:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:21:16.679558725 +0000 UTC m=+343.033135864" watchObservedRunningTime="2026-07-24 20:21:16.682237929 +0000 UTC m=+343.035815068"
Jul 24 20:21:21.569230 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:21.569196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:21.670337 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:21.670309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:25.653186 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:25.653111    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-779f8ddb8c-qbk9s"
Jul 24 20:21:27.665386 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:27.665357    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl"
Jul 24 20:21:29.653252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:29.653191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:29.779676 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:29.779639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.241510 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.241473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.290880 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.290847    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"]
Jul 24 20:21:30.298362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.298339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.301007 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.300987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 20:21:30.301305 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.301290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 20:21:30.302402 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302381    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7qqmc\""
Jul 24 20:21:30.302535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 20:21:30.302535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 20:21:30.302789 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tmp\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.302830 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.302901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.302888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m24p\" (UniqueName: \"kubernetes.io/projected/f33e003d-1eb1-4235-8869-a9ed5a34f68e-kube-api-access-2m24p\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.311711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.311688    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"]
Jul 24 20:21:30.403756 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.403725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tmp\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.403756 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.403760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.403991 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.403826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m24p\" (UniqueName: \"kubernetes.io/projected/f33e003d-1eb1-4235-8869-a9ed5a34f68e-kube-api-access-2m24p\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.403991 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:21:30.403925    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 24 20:21:30.403991 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:21:30.403982    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs podName:f33e003d-1eb1-4235-8869-a9ed5a34f68e nodeName:}" failed. No retries permitted until 2026-07-24 20:21:30.9039668 +0000 UTC m=+357.257543918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs") pod "kube-auth-proxy-68fdc99b6-nrz9s" (UID: "f33e003d-1eb1-4235-8869-a9ed5a34f68e") : secret "kube-auth-proxy-tls" not found
Jul 24 20:21:30.406148 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.406123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tmp\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.424846 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.424819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m24p\" (UniqueName: \"kubernetes.io/projected/f33e003d-1eb1-4235-8869-a9ed5a34f68e-kube-api-access-2m24p\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.509095 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.509067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.607141 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.607047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.702838 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.702803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.801127 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.801044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:30.907218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.907183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.909631 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.909589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33e003d-1eb1-4235-8869-a9ed5a34f68e-tls-certs\") pod \"kube-auth-proxy-68fdc99b6-nrz9s\" (UID: \"f33e003d-1eb1-4235-8869-a9ed5a34f68e\") " pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:30.913686 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:30.913662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:31.208103 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.208026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"
Jul 24 20:21:31.350691 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.350669    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s"]
Jul 24 20:21:31.353402 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:21:31.353373    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33e003d_1eb1_4235_8869_a9ed5a34f68e.slice/crio-8a5f41d8432a72bcfbbb37dbd22fea70318bd0feda96e249ae613724d0946f0b WatchSource:0}: Error finding container 8a5f41d8432a72bcfbbb37dbd22fea70318bd0feda96e249ae613724d0946f0b: Status 404 returned error can't find the container with id 8a5f41d8432a72bcfbbb37dbd22fea70318bd0feda96e249ae613724d0946f0b
Jul 24 20:21:31.452053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.452019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:31.725484 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.725451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s" event={"ID":"f33e003d-1eb1-4235-8869-a9ed5a34f68e","Type":"ContainerStarted","Data":"8a5f41d8432a72bcfbbb37dbd22fea70318bd0feda96e249ae613724d0946f0b"}
Jul 24 20:21:31.813078 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.813042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:31.879780 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:31.879744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:32.802732 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:32.802695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:34.739349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:34.739311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s" event={"ID":"f33e003d-1eb1-4235-8869-a9ed5a34f68e","Type":"ContainerStarted","Data":"9c30b87aa870a343494dc5f7ee4ad58c7fcf4531d6e5162978d8c2878db6f707"}
Jul 24 20:21:34.769976 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:34.769928    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68fdc99b6-nrz9s" podStartSLOduration=2.447186514 podStartE2EDuration="4.769914192s" totalImagesPullingTime="2.322727678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:21:34.769476592 +0000 UTC m=+361.123053760" watchObservedRunningTime="2026-07-24 20:21:34.769914192 +0000 UTC m=+361.123491332"
Jul 24 20:21:37.477889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:37.477854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:37.578122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:37.578089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:37.857239 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:37.857208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:38.011155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:38.011119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:38.114800 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:38.114720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:40.630971 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:40.630934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:40.732000 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:40.731963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:41.277734 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:41.277700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:41.457972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:41.457939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:41.896774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:41.896738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:41.998333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:41.998299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:21:42.114581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:21:42.114548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:22:27.357844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:22:27.357813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:22:27.417298 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:22:27.417268    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:22:31.150090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:22:31.150047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:22:31.277032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:22:31.276797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:22:31.356046 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:22:31.355977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:01.992923 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:01.992885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:02.610519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:02.610483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:07.427427 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:07.427396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:09.466120 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:09.466087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:10.316290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:10.316260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:10.414056 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:10.414023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:10.517405 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:10.517371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:11.244763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:11.244732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:13.419929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:13.419898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:13.894381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:13.894350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:17.079381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:17.079346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:19.024957 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:19.024926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:19.123605 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:19.123575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:19.421963 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:19.421886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:19.524151 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:19.524115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:21.378450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:21.378419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:21.477883 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:21.477852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:21.583115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:21.583087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:21.681661 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:21.681576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:22.825851 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:22.825823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:22.877815 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:22.877786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:26.634581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.634540    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:26.736723 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.736692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:26.895116 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.895039    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"]
Jul 24 20:23:26.897268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.897251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:26.901022 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.901001    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 20:23:26.901022 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.901019    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 20:23:26.901294 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.901273    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 20:23:26.902247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.902226    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 20:23:26.902336 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.902264    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9vz6v\""
Jul 24 20:23:26.905976 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.905953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:26.909566 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.909545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"]
Jul 24 20:23:26.998498 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:26.998470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:27.089852 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.089822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.090003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.089875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.090003 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.089912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjwhl\" (UniqueName: \"kubernetes.io/projected/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-kube-api-access-jjwhl\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.191328 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.191247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.191328 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.191284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.191328 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.191313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjwhl\" (UniqueName: \"kubernetes.io/projected/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-kube-api-access-jjwhl\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.191905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.191882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.193812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.193792    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.199663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.199631    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjwhl\" (UniqueName: \"kubernetes.io/projected/3f1a5f92-31af-437c-bbb2-b4db6b8318e7-kube-api-access-jjwhl\") pod \"kuadrant-console-plugin-66b5774985-tfjtb\" (UID: \"3f1a5f92-31af-437c-bbb2-b4db6b8318e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.206461 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.206440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"
Jul 24 20:23:27.338958 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:27.338934    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb"]
Jul 24 20:23:27.340962 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:23:27.340937    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f1a5f92_31af_437c_bbb2_b4db6b8318e7.slice/crio-e4b39b44639a5f514d3ceedad91474939c4ac70d4160c66bbac2baad532c606d WatchSource:0}: Error finding container e4b39b44639a5f514d3ceedad91474939c4ac70d4160c66bbac2baad532c606d: Status 404 returned error can't find the container with id e4b39b44639a5f514d3ceedad91474939c4ac70d4160c66bbac2baad532c606d
Jul 24 20:23:28.253899 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:28.253853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb" event={"ID":"3f1a5f92-31af-437c-bbb2-b4db6b8318e7","Type":"ContainerStarted","Data":"e4b39b44639a5f514d3ceedad91474939c4ac70d4160c66bbac2baad532c606d"}
Jul 24 20:23:30.272561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:30.272519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:30.332876 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:30.332821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:31.615018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:31.614980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:38.248315 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:38.248269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:38.351809 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:38.351774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:38.843906 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:38.843873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:38.951531 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:38.951490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:41.828367 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:41.828328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:41.879427 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:41.879398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:45.404653 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:45.404618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:45.471271 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:45.471218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:46.438385 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:46.438337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:46.538916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:46.538877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:49.013107 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:49.013066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:49.113222 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:49.113191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:23:50.369664 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:50.369631    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb" event={"ID":"3f1a5f92-31af-437c-bbb2-b4db6b8318e7","Type":"ContainerStarted","Data":"95250786d9673261236bb8bec02e18cd027f4cc47b89e1dba3619f30c54afd0f"}
Jul 24 20:23:50.387002 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:23:50.386953    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-tfjtb" podStartSLOduration=2.154829765 podStartE2EDuration="24.386939148s" totalImagesPullingTime="22.232109383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:23:50.385531794 +0000 UTC m=+496.739108939" watchObservedRunningTime="2026-07-24 20:23:50.386939148 +0000 UTC m=+496.740516305"
Jul 24 20:24:06.095434 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:06.095406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:06.246988 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:06.246955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:19.804752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:19.804722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:19.900517 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:19.900472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:20.000433 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.000401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:20.257911 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.257832    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:24:20.326526 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.326497    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:24:20.326526 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.326533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:24:20.326758 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.326630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.329343 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.329319    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 20:24:20.381667 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.381634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wm67\" (UniqueName: \"kubernetes.io/projected/25b90d72-2337-48bb-90bb-ffd33308f2d8-kube-api-access-8wm67\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.381805 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.381677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/25b90d72-2337-48bb-90bb-ffd33308f2d8-config-file\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.482390 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.482358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wm67\" (UniqueName: \"kubernetes.io/projected/25b90d72-2337-48bb-90bb-ffd33308f2d8-kube-api-access-8wm67\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.482591 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.482412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/25b90d72-2337-48bb-90bb-ffd33308f2d8-config-file\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.483074 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.483051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/25b90d72-2337-48bb-90bb-ffd33308f2d8-config-file\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.490983 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.490963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wm67\" (UniqueName: \"kubernetes.io/projected/25b90d72-2337-48bb-90bb-ffd33308f2d8-kube-api-access-8wm67\") pod \"limitador-limitador-54d6d988d5-zbrfp\" (UID: \"25b90d72-2337-48bb-90bb-ffd33308f2d8\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.607086 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.607056    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:20.636977 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.636949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:20.665431 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.665400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:20.771149 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:20.771127    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:24:20.773691 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:24:20.773662    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b90d72_2337_48bb_90bb_ffd33308f2d8.slice/crio-4f0bc66c107e2d0b254b15199660681db50f2ac0b6f7a0a10a946de9f3dcef0f WatchSource:0}: Error finding container 4f0bc66c107e2d0b254b15199660681db50f2ac0b6f7a0a10a946de9f3dcef0f: Status 404 returned error can't find the container with id 4f0bc66c107e2d0b254b15199660681db50f2ac0b6f7a0a10a946de9f3dcef0f
Jul 24 20:24:21.510830 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:21.510716    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp" event={"ID":"25b90d72-2337-48bb-90bb-ffd33308f2d8","Type":"ContainerStarted","Data":"4f0bc66c107e2d0b254b15199660681db50f2ac0b6f7a0a10a946de9f3dcef0f"}
Jul 24 20:24:23.520760 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:23.520728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp" event={"ID":"25b90d72-2337-48bb-90bb-ffd33308f2d8","Type":"ContainerStarted","Data":"91b3eb89298b5c2bdf00349708b675be287a748b5f5ba45a43bb499c22a3daa3"}
Jul 24 20:24:23.521079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:23.520807    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:23.536720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:23.536675    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp" podStartSLOduration=0.900752655 podStartE2EDuration="3.536660974s" totalImagesPullingTime="2.635908319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:24:23.535332206 +0000 UTC m=+529.888909363" watchObservedRunningTime="2026-07-24 20:24:23.536660974 +0000 UTC m=+529.890238114"
Jul 24 20:24:25.583960 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:25.583886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:34.527159 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:34.527127    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"
Jul 24 20:24:39.251790 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:39.251751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:39.346529 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:39.346501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:39.736351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:39.736313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:52.253061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:52.253030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:52.728052 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:52.728011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:52.822604 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:52.822570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:59.770451 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:59.770419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:59.871143 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:59.871107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:24:59.965398 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:24:59.965363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:00.023792 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:00.023706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:02.911252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:02.911219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:03.018855 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:03.018800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:12.297995 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:12.297964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:12.399101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:12.399066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:35.474979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:35.474942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:36.544969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.544932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:36.640834 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.640809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:36.713544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.713509    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-g72sw"]
Jul 24 20:25:36.716197 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.716181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.719699 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.719678    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 20:25:36.719802 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.719786    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-drjxq\""
Jul 24 20:25:36.734987 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.734967    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-g72sw"]
Jul 24 20:25:36.749612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.749578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:36.755873 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.755853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dpz\" (UniqueName: \"kubernetes.io/projected/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-kube-api-access-94dpz\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.755945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.755896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.790706 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.790675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:25:36.821091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.821032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:36.857615 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.857561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dpz\" (UniqueName: \"kubernetes.io/projected/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-kube-api-access-94dpz\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.857789 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.857662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.858552 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.858520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:36.868578 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:36.868556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dpz\" (UniqueName: \"kubernetes.io/projected/30801f6e-2fa9-4bfd-b814-0a81cc5c33a2-kube-api-access-94dpz\") pod \"payload-processing-5cc6cd444b-g72sw\" (UID: \"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:37.026533 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:37.026483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:37.159426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:37.159405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-g72sw"]
Jul 24 20:25:37.161220 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:25:37.161182    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30801f6e_2fa9_4bfd_b814_0a81cc5c33a2.slice/crio-0518da67ec0672485a8ada50e58478d13fb340f7b69e512df760ab66c53538d0 WatchSource:0}: Error finding container 0518da67ec0672485a8ada50e58478d13fb340f7b69e512df760ab66c53538d0: Status 404 returned error can't find the container with id 0518da67ec0672485a8ada50e58478d13fb340f7b69e512df760ab66c53538d0
Jul 24 20:25:37.163108 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:37.163090    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:25:37.866716 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:37.866678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw" event={"ID":"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2","Type":"ContainerStarted","Data":"0518da67ec0672485a8ada50e58478d13fb340f7b69e512df760ab66c53538d0"}
Jul 24 20:25:38.874013 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:38.873976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw" event={"ID":"30801f6e-2fa9-4bfd-b814-0a81cc5c33a2","Type":"ContainerStarted","Data":"24424f0ab7bd59817adb5b9343a39b8588aee3baf4ed59b32412494129d43a96"}
Jul 24 20:25:38.874395 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:38.874166    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:38.890740 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:38.890697    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw" podStartSLOduration=1.373527965 podStartE2EDuration="2.890682255s" totalImagesPullingTime="1.51715429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:25:38.888556233 +0000 UTC m=+605.242133394" watchObservedRunningTime="2026-07-24 20:25:38.890682255 +0000 UTC m=+605.244259395"
Jul 24 20:25:48.208421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:48.208382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:48.308231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:48.308197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:49.881181 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:49.881151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-g72sw"
Jul 24 20:25:51.544246 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:51.544211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:51.799895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:51.799826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:51.873928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:51.873898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:25:52.124056 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:52.123975    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:25:59.525954 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:25:59.525923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:00.302674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:00.302645    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:26:10.692419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.692384    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"]
Jul 24 20:26:10.695888 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.695869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.698651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.698574    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 20:26:10.698651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.698578    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q65fn\""
Jul 24 20:26:10.698651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.698576    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 20:26:10.699797 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.699784    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 20:26:10.704701 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.704682    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"]
Jul 24 20:26:10.794675 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.794643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:10.853873 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97391990-abb7-4ec5-b4bd-cd31b4467297-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.854008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.854008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24wd9\" (UniqueName: \"kubernetes.io/projected/97391990-abb7-4ec5-b4bd-cd31b4467297-kube-api-access-24wd9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.854008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.854008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.854008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.853984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97391990-abb7-4ec5-b4bd-cd31b4467297-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24wd9\" (UniqueName: \"kubernetes.io/projected/97391990-abb7-4ec5-b4bd-cd31b4467297-kube-api-access-24wd9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955743 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955798 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.955841 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.955802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.957804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.957784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97391990-abb7-4ec5-b4bd-cd31b4467297-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.957916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.957899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97391990-abb7-4ec5-b4bd-cd31b4467297-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:10.963679 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:10.963653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24wd9\" (UniqueName: \"kubernetes.io/projected/97391990-abb7-4ec5-b4bd-cd31b4467297-kube-api-access-24wd9\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9bsth\" (UID: \"97391990-abb7-4ec5-b4bd-cd31b4467297\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:11.007724 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:11.007692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:11.144346 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:11.144323    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"]
Jul 24 20:26:11.146733 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:26:11.146694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97391990_abb7_4ec5_b4bd_cd31b4467297.slice/crio-45836de936145fb6c97fe1f44b4aed5951349b243c90285b1b3be00b74d90124 WatchSource:0}: Error finding container 45836de936145fb6c97fe1f44b4aed5951349b243c90285b1b3be00b74d90124: Status 404 returned error can't find the container with id 45836de936145fb6c97fe1f44b4aed5951349b243c90285b1b3be00b74d90124
Jul 24 20:26:11.998962 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:11.998929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:26:12.024075 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:12.024038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth" event={"ID":"97391990-abb7-4ec5-b4bd-cd31b4467297","Type":"ContainerStarted","Data":"45836de936145fb6c97fe1f44b4aed5951349b243c90285b1b3be00b74d90124"}
Jul 24 20:26:13.029722 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:13.029688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth" event={"ID":"97391990-abb7-4ec5-b4bd-cd31b4467297","Type":"ContainerStarted","Data":"169b0f0e7c14fd90446b7410b152cb4ca0968c48c503e1337f54366ba4c7350e"}
Jul 24 20:26:13.048332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:13.048287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth" podStartSLOduration=2.032728326 podStartE2EDuration="3.048271587s" totalImagesPullingTime="1.015543261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:26:13.045973642 +0000 UTC m=+639.399550782" watchObservedRunningTime="2026-07-24 20:26:13.048271587 +0000 UTC m=+639.401848778"
Jul 24 20:26:16.107821 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:16.107787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:16.164690 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:16.164653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:20.927399 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:20.927361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:21.008543 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:21.008510    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:21.008759 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:21.008568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:21.021050 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:21.021024    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:21.079263 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:21.079230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9bsth"
Jul 24 20:26:22.506967 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:22.506933    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:26:34.763207 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:34.763173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:39.649079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:39.649047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:42.295539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:42.295507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:26:51.099101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.099068    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"]
Jul 24 20:26:51.105252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.105226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.107931 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.107908    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 20:26:51.111612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.111565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"]
Jul 24 20:26:51.129970 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.129934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:26:51.173017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.172977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.173017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.173021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9mlp\" (UniqueName: \"kubernetes.io/projected/f40e76f5-2a6e-428f-a89a-a641a04ed79f-kube-api-access-h9mlp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.173239 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.173056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.173239 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.173088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.173239 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.173107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.173239 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.173128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274368 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9mlp\" (UniqueName: \"kubernetes.io/projected/f40e76f5-2a6e-428f-a89a-a641a04ed79f-kube-api-access-h9mlp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274937 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.274937 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.274933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.275211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.275186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.276650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.276630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f40e76f5-2a6e-428f-a89a-a641a04ed79f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.276980 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.276963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f40e76f5-2a6e-428f-a89a-a641a04ed79f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.282577 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.282554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9mlp\" (UniqueName: \"kubernetes.io/projected/f40e76f5-2a6e-428f-a89a-a641a04ed79f-kube-api-access-h9mlp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-glb2f\" (UID: \"f40e76f5-2a6e-428f-a89a-a641a04ed79f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.414859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.414775    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:26:51.546000 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:51.545843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"]
Jul 24 20:26:51.548234 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:26:51.548211    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf40e76f5_2a6e_428f_a89a_a641a04ed79f.slice/crio-10c2b74231bbecc5f9f46844f7bdb0d562c0138e21100f092a2fb661abf739ce WatchSource:0}: Error finding container 10c2b74231bbecc5f9f46844f7bdb0d562c0138e21100f092a2fb661abf739ce: Status 404 returned error can't find the container with id 10c2b74231bbecc5f9f46844f7bdb0d562c0138e21100f092a2fb661abf739ce
Jul 24 20:26:52.209312 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:52.209263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f" event={"ID":"f40e76f5-2a6e-428f-a89a-a641a04ed79f","Type":"ContainerStarted","Data":"48d96ba898cc6124a21b4f3a75c40acc78b483a99cffcb9e287eb1c1b3b3c716"}
Jul 24 20:26:52.209821 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:52.209321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f" event={"ID":"f40e76f5-2a6e-428f-a89a-a641a04ed79f","Type":"ContainerStarted","Data":"10c2b74231bbecc5f9f46844f7bdb0d562c0138e21100f092a2fb661abf739ce"}
Jul 24 20:26:54.590284 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:54.590232    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f" podStartSLOduration=3.413129946 podStartE2EDuration="3.590214282s" totalImagesPullingTime="177.084336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:26:52.227309545 +0000 UTC m=+678.580886686" watchObservedRunningTime="2026-07-24 20:26:54.590214282 +0000 UTC m=+680.943791422"
Jul 24 20:26:54.591625 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:26:54.591607    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:27:01.415209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:01.415169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:27:01.415209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:01.415216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:27:01.427543 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:01.427518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:27:02.263380 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:02.263346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-glb2f"
Jul 24 20:27:33.277776 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:33.277689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:27:33.395498 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:33.395467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:27:34.442564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:34.442518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:27:34.541855 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:27:34.541826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:28:03.309419 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:28:03.309382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:28:13.296755 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:28:13.296721    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:28:23.704624 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:28:23.704564    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:29:12.600948 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:29:12.600912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:29:12.689350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:29:12.689287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:29:38.813801 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:29:38.813760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:29:40.333483 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:29:40.333452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:05.993017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:05.992985    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:30:13.700981 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:13.700948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:30:15.968837 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:15.968801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:16.067114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:16.067081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:16.195081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:16.195048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:21.930465 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:21.930430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:22.032377 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:22.032345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:22.130924 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:22.130887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:22.233085 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:22.233010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:22.330375 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:22.330338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:27.955776 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:27.955695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:28.075607 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:28.075573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:30:46.692201 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:46.692164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:30:56.093692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:30:56.093644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:05.405387 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:05.405350    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:14.796205 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:14.796169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:25.399556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:25.399512    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:35.398036 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:35.398000    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:46.393653 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:46.393621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:56.092167 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:56.092129    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:31:58.558137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:31:58.558095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:32:18.396368 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:32:18.396331    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:32:29.595737 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:32:29.595698    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:32:53.697864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:32:53.697832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:03.605727 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:03.605687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:33:03.660796 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:03.660746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:33:04.293117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:04.293087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:19.402916 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:19.402881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:28.396318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:28.396241    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:40.391926 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:40.391891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:49.402829 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:49.402795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:33:59.091628 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:33:59.091578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:34:55.258787 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:34:55.258706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:35:48.197057 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:35:48.197024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:35:59.797698 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:35:59.797660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:36:08.791860 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:36:08.791820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:36:20.601783 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:36:20.601743    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:36:30.489293 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:36:30.489214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:36:42.295106 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:36:42.295077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:36:50.999827 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:36:50.999789    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:37:02.596788 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:37:02.596737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:37:11.892711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:37:11.892675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:37:21.943208 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:37:21.943172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:37:32.890497 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:37:32.890464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:37:54.596875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:37:54.596841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:38:04.090562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:04.090515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:38:12.455973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:12.455939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:38:16.396858 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:16.396825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:38:20.208394 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:20.208359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:38:31.788268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:31.788232    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:38:43.095063 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:43.095020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:38:44.591163 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:44.591127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:38:46.989421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:46.989386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:38:59.793862 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:38:59.793821    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:01.542739 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:01.542695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:39:06.397518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:06.397481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:22.692378 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:22.692300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:35.997646 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:35.997551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:44.788863 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:44.788826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:50.290551 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:50.290511    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:39:59.092644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:39:59.092606    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:40:08.789124 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:40:08.789083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:40:21.393449 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:40:21.393414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:40:30.489277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:40:30.489240    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:40:40.391913 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:40:40.391876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:40:51.894230 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:40:51.894193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:03.896766 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:03.896731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:11.838268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:11.838233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:15.449536 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:15.449506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:41:20.881633 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:20.881582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:34.181143 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:34.181113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:47.488115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:47.488075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:41:48.515905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:48.515829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:41:59.689011 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:41:59.688977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:42:08.990398 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:08.990363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:42:10.160786 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:10.160720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:13.607529 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:13.607490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:13.707203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:13.707164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:14.150720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:14.150688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:18.583572 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:18.583538    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:42:19.874588 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:19.874543    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:19.934574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:19.934534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:20.120910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:20.120878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:20.454048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:20.454022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:20.628074 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:20.628040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:22.718726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:22.718690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:22.818731 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:22.818701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:22.908430 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:22.908403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:23.118048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.118014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:23.504072 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.503992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:23.595796 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.595766    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:23.632883 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.632853    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:42:23.636464 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.636443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.638893 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.638871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8c1aeab3-dockercfg-trf7c\""
Jul 24 20:42:23.639131 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.639105    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8c1aeab3\""
Jul 24 20:42:23.647144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.647123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:42:23.761021 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.760993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96s64\" (UniqueName: \"kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.761409 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.761037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.862475 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.862442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96s64\" (UniqueName: \"kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.862645 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.862488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.863195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.863176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.870980 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.870958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96s64\" (UniqueName: \"kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64\") pod \"payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") " pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:23.947708 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:23.947680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:24.080253 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:24.080223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:42:24.081789 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:42:24.081762    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd02a43_d1ce_4e23_b861_fe8b90f45a14.slice/crio-43d07a4c3ab355ce714f65f63359eae855ec6e25614e9abdd445c6c483a558eb WatchSource:0}: Error finding container 43d07a4c3ab355ce714f65f63359eae855ec6e25614e9abdd445c6c483a558eb: Status 404 returned error can't find the container with id 43d07a4c3ab355ce714f65f63359eae855ec6e25614e9abdd445c6c483a558eb
Jul 24 20:42:24.488053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:24.487964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" event={"ID":"efd02a43-d1ce-4e23-b861-fe8b90f45a14","Type":"ContainerStarted","Data":"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"}
Jul 24 20:42:24.488053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:24.488024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" event={"ID":"efd02a43-d1ce-4e23-b861-fe8b90f45a14","Type":"ContainerStarted","Data":"43d07a4c3ab355ce714f65f63359eae855ec6e25614e9abdd445c6c483a558eb"}
Jul 24 20:42:24.488053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:24.488052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:24.504124 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:24.504079    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" podStartSLOduration=1.504065072 podStartE2EDuration="1.504065072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:42:24.502836084 +0000 UTC m=+1610.856413245" watchObservedRunningTime="2026-07-24 20:42:24.504065072 +0000 UTC m=+1610.857642229"
Jul 24 20:42:29.483741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:29.483707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:42:35.494947 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:35.494915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:58.983206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:58.983169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:59.006361 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.006326    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:42:59.006677 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.006625    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" podUID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" containerName="payload-processing" containerID="cri-o://4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535" gracePeriod=30
Jul 24 20:42:59.081410 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.081370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:59.130292 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.130264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:42:59.260700 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.260675    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:59.374920 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.374891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume\") pod \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") "
Jul 24 20:42:59.375083 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.374987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96s64\" (UniqueName: \"kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64\") pod \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\" (UID: \"efd02a43-d1ce-4e23-b861-fe8b90f45a14\") "
Jul 24 20:42:59.375315 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.375293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efd02a43-d1ce-4e23-b861-fe8b90f45a14" (UID: "efd02a43-d1ce-4e23-b861-fe8b90f45a14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:42:59.377238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.377220    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64" (OuterVolumeSpecName: "kube-api-access-96s64") pod "efd02a43-d1ce-4e23-b861-fe8b90f45a14" (UID: "efd02a43-d1ce-4e23-b861-fe8b90f45a14"). InnerVolumeSpecName "kube-api-access-96s64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:42:59.475845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.475813    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efd02a43-d1ce-4e23-b861-fe8b90f45a14-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:42:59.475845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.475841    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96s64\" (UniqueName: \"kubernetes.io/projected/efd02a43-d1ce-4e23-b861-fe8b90f45a14-kube-api-access-96s64\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:42:59.663049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.663015    2577 generic.go:356] "Generic (PLEG): container finished" podID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" containerID="4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535" exitCode=0
Jul 24 20:42:59.663233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.663059    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"
Jul 24 20:42:59.663233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.663100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" event={"ID":"efd02a43-d1ce-4e23-b861-fe8b90f45a14","Type":"ContainerDied","Data":"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"}
Jul 24 20:42:59.663233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.663150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt" event={"ID":"efd02a43-d1ce-4e23-b861-fe8b90f45a14","Type":"ContainerDied","Data":"43d07a4c3ab355ce714f65f63359eae855ec6e25614e9abdd445c6c483a558eb"}
Jul 24 20:42:59.663233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.663173    2577 scope.go:117] "RemoveContainer" containerID="4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"
Jul 24 20:42:59.672087 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.672071    2577 scope.go:117] "RemoveContainer" containerID="4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"
Jul 24 20:42:59.672325 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:42:59.672307    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535\": container with ID starting with 4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535 not found: ID does not exist" containerID="4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"
Jul 24 20:42:59.672408 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.672341    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535"} err="failed to get container status \"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535\": rpc error: code = NotFound desc = could not find container \"4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535\": container with ID starting with 4474ddd774a0d0b97a3d25518f3d2a3b9f82e2ae4a525cd247f12c7e6cf48535 not found: ID does not exist"
Jul 24 20:42:59.683135 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.683109    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:42:59.687398 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:42:59.687378    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c1aeab3-5df9b45584-lv6zt"]
Jul 24 20:43:00.153107 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:00.153075    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" path="/var/lib/kubelet/pods/efd02a43-d1ce-4e23-b861-fe8b90f45a14/volumes"
Jul 24 20:43:01.591059 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:01.591027    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:43:03.685445 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:03.685410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:03.829445 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:03.829390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:04.111256 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:04.111222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:04.366142 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:04.366062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:04.458461 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:04.458423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:04.594823 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:04.594794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:04.941833 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:04.941799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:05.187838 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:05.187807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:07.503773 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:07.503739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:07.691129 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:07.691095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:08.122871 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.122836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:08.219622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.219560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:08.228067 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.228043    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:08.228439 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.228427    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" containerName="payload-processing"
Jul 24 20:43:08.228481 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.228442    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" containerName="payload-processing"
Jul 24 20:43:08.228517 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.228506    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efd02a43-d1ce-4e23-b861-fe8b90f45a14" containerName="payload-processing"
Jul 24 20:43:08.231301 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.231286    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.234112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.234096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-42b8f369-dockercfg-jzwnf\""
Jul 24 20:43:08.234366 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.234355    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-42b8f369\""
Jul 24 20:43:08.240045 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.240023    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:08.263280 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.263255    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:08.266893 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.266876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.276342 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.276319    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:08.353448 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.353412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7cfc\" (UniqueName: \"kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.353642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.353456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld6nf\" (UniqueName: \"kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.353642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.353539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.353642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.353587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.454523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.454444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7cfc\" (UniqueName: \"kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.454523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.454483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld6nf\" (UniqueName: \"kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.454761 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.454539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.454761 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.454585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.455289 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.455269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.455329 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.455269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.462404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.462379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld6nf\" (UniqueName: \"kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf\") pod \"payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") " pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.462653 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.462633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7cfc\" (UniqueName: \"kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc\") pod \"payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.542504 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.542471    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:08.578852 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.578822    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:08.691065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.691031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:08.693139 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:43:08.693101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0314d62b_8300_4226_b3f2_1d77768ae56f.slice/crio-be4bef13103eed1d2d5c3d37af651501494d29ad32c30db0f2a07343b843a775 WatchSource:0}: Error finding container be4bef13103eed1d2d5c3d37af651501494d29ad32c30db0f2a07343b843a775: Status 404 returned error can't find the container with id be4bef13103eed1d2d5c3d37af651501494d29ad32c30db0f2a07343b843a775
Jul 24 20:43:08.710182 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.710152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" event={"ID":"0314d62b-8300-4226-b3f2-1d77768ae56f","Type":"ContainerStarted","Data":"be4bef13103eed1d2d5c3d37af651501494d29ad32c30db0f2a07343b843a775"}
Jul 24 20:43:08.734241 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:08.734219    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:08.740719 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:43:08.740694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7889e4_50f9_49da_8b5d_c95d36d7887b.slice/crio-156c50bae1bea51e4f0c22b4639e10c2c8f113affc9f3b0e19124dcfcad3efc1 WatchSource:0}: Error finding container 156c50bae1bea51e4f0c22b4639e10c2c8f113affc9f3b0e19124dcfcad3efc1: Status 404 returned error can't find the container with id 156c50bae1bea51e4f0c22b4639e10c2c8f113affc9f3b0e19124dcfcad3efc1
Jul 24 20:43:09.718304 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.718268    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" event={"ID":"0314d62b-8300-4226-b3f2-1d77768ae56f","Type":"ContainerStarted","Data":"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"}
Jul 24 20:43:09.718851 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.718372    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:09.719910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.719883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" event={"ID":"0a7889e4-50f9-49da-8b5d-c95d36d7887b","Type":"ContainerStarted","Data":"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"}
Jul 24 20:43:09.720039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.719929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" event={"ID":"0a7889e4-50f9-49da-8b5d-c95d36d7887b","Type":"ContainerStarted","Data":"156c50bae1bea51e4f0c22b4639e10c2c8f113affc9f3b0e19124dcfcad3efc1"}
Jul 24 20:43:09.720039 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.719959    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:09.734407 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.734359    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" podStartSLOduration=1.734344388 podStartE2EDuration="1.734344388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:43:09.733916065 +0000 UTC m=+1656.087493206" watchObservedRunningTime="2026-07-24 20:43:09.734344388 +0000 UTC m=+1656.087921527"
Jul 24 20:43:09.749812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:09.749769    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" podStartSLOduration=1.749755238 podStartE2EDuration="1.749755238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:43:09.74828844 +0000 UTC m=+1656.101865583" watchObservedRunningTime="2026-07-24 20:43:09.749755238 +0000 UTC m=+1656.103332377"
Jul 24 20:43:14.289544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:14.289508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:43:20.727625 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:20.727570    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:20.728117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:20.727766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:43.049743 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.049711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:43.083901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.083867    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:43.084148 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.084121    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" podUID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" containerName="payload-processing" containerID="cri-o://d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c" gracePeriod=30
Jul 24 20:43:43.096053 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.096021    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:43.097008 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.096247    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" podUID="0314d62b-8300-4226-b3f2-1d77768ae56f" containerName="payload-pre-processing" containerID="cri-o://45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf" gracePeriod=30
Jul 24 20:43:43.343675 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.343649    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:43.367705 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.367681    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld6nf\" (UniqueName: \"kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf\") pod \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") "
Jul 24 20:43:43.367843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.367783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume\") pod \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\" (UID: \"0a7889e4-50f9-49da-8b5d-c95d36d7887b\") "
Jul 24 20:43:43.368244 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.368178    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a7889e4-50f9-49da-8b5d-c95d36d7887b" (UID: "0a7889e4-50f9-49da-8b5d-c95d36d7887b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:43:43.369798 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.369774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf" (OuterVolumeSpecName: "kube-api-access-ld6nf") pod "0a7889e4-50f9-49da-8b5d-c95d36d7887b" (UID: "0a7889e4-50f9-49da-8b5d-c95d36d7887b"). InnerVolumeSpecName "kube-api-access-ld6nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:43:43.370035 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.370012    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:43.468903 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.468877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume\") pod \"0314d62b-8300-4226-b3f2-1d77768ae56f\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") "
Jul 24 20:43:43.469078 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.468971    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7cfc\" (UniqueName: \"kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc\") pod \"0314d62b-8300-4226-b3f2-1d77768ae56f\" (UID: \"0314d62b-8300-4226-b3f2-1d77768ae56f\") "
Jul 24 20:43:43.469309 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.469292    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a7889e4-50f9-49da-8b5d-c95d36d7887b-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:43:43.469354 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.469323    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld6nf\" (UniqueName: \"kubernetes.io/projected/0a7889e4-50f9-49da-8b5d-c95d36d7887b-kube-api-access-ld6nf\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:43:43.469354 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.469290    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0314d62b-8300-4226-b3f2-1d77768ae56f" (UID: "0314d62b-8300-4226-b3f2-1d77768ae56f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:43:43.471195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.471167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc" (OuterVolumeSpecName: "kube-api-access-h7cfc") pod "0314d62b-8300-4226-b3f2-1d77768ae56f" (UID: "0314d62b-8300-4226-b3f2-1d77768ae56f"). InnerVolumeSpecName "kube-api-access-h7cfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:43:43.570633 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.570575    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0314d62b-8300-4226-b3f2-1d77768ae56f-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:43:43.570633 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.570629    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7cfc\" (UniqueName: \"kubernetes.io/projected/0314d62b-8300-4226-b3f2-1d77768ae56f-kube-api-access-h7cfc\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:43:43.883724 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.883688    2577 generic.go:356] "Generic (PLEG): container finished" podID="0314d62b-8300-4226-b3f2-1d77768ae56f" containerID="45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf" exitCode=0
Jul 24 20:43:43.883938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.883737    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"
Jul 24 20:43:43.883938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.883786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" event={"ID":"0314d62b-8300-4226-b3f2-1d77768ae56f","Type":"ContainerDied","Data":"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"}
Jul 24 20:43:43.883938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.883852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl" event={"ID":"0314d62b-8300-4226-b3f2-1d77768ae56f","Type":"ContainerDied","Data":"be4bef13103eed1d2d5c3d37af651501494d29ad32c30db0f2a07343b843a775"}
Jul 24 20:43:43.883938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.883881    2577 scope.go:117] "RemoveContainer" containerID="45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"
Jul 24 20:43:43.885099 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.885076    2577 generic.go:356] "Generic (PLEG): container finished" podID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" containerID="d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c" exitCode=0
Jul 24 20:43:43.885162 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.885117    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"
Jul 24 20:43:43.885216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.885165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" event={"ID":"0a7889e4-50f9-49da-8b5d-c95d36d7887b","Type":"ContainerDied","Data":"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"}
Jul 24 20:43:43.885265 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.885219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79" event={"ID":"0a7889e4-50f9-49da-8b5d-c95d36d7887b","Type":"ContainerDied","Data":"156c50bae1bea51e4f0c22b4639e10c2c8f113affc9f3b0e19124dcfcad3efc1"}
Jul 24 20:43:43.892757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.892742    2577 scope.go:117] "RemoveContainer" containerID="45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"
Jul 24 20:43:43.893053 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:43:43.893029    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf\": container with ID starting with 45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf not found: ID does not exist" containerID="45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"
Jul 24 20:43:43.893101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.893066    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf"} err="failed to get container status \"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf\": rpc error: code = NotFound desc = could not find container \"45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf\": container with ID starting with 45cc1184e29f134d25d49c7bca6526ca13b78e10704a2860933c54e6eb760caf not found: ID does not exist"
Jul 24 20:43:43.893101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.893090    2577 scope.go:117] "RemoveContainer" containerID="d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"
Jul 24 20:43:43.901304 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.901287    2577 scope.go:117] "RemoveContainer" containerID="d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"
Jul 24 20:43:43.901548 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:43:43.901529    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c\": container with ID starting with d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c not found: ID does not exist" containerID="d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"
Jul 24 20:43:43.901622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.901556    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c"} err="failed to get container status \"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c\": rpc error: code = NotFound desc = could not find container \"d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c\": container with ID starting with d4ec4f6a37447de0ba95d20438a57b3cf9edbe42175a034595240d0c1ed8018c not found: ID does not exist"
Jul 24 20:43:43.909041 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.909020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:43.912214 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.912195    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-42b8f369-868457b7d5-p5q79"]
Jul 24 20:43:43.922448 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.922429    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:43.925660 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:43.925630    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-42b8f369-675b4b6988-h9jbl"]
Jul 24 20:43:44.154961 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:44.154875    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0314d62b-8300-4226-b3f2-1d77768ae56f" path="/var/lib/kubelet/pods/0314d62b-8300-4226-b3f2-1d77768ae56f/volumes"
Jul 24 20:43:44.155403 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:44.155378    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" path="/var/lib/kubelet/pods/0a7889e4-50f9-49da-8b5d-c95d36d7887b/volumes"
Jul 24 20:43:44.201807 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:44.201774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:46.091471 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:46.091437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:43:47.676538 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:47.676506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:47.828216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:47.828189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:48.094012 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:48.093980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:48.369864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:48.369777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:48.682574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:48.682486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:49.038949 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:49.038877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:51.212272 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:51.212247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:51.284185 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:51.284146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:51.387050 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:51.387022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:51.584816 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:51.584774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:51.929560 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:51.929481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:52.021534 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.021498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:43:52.046116 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046085    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:43:52.046483 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046471    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" containerName="payload-processing"
Jul 24 20:43:52.046535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046485    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" containerName="payload-processing"
Jul 24 20:43:52.046535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046509    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0314d62b-8300-4226-b3f2-1d77768ae56f" containerName="payload-pre-processing"
Jul 24 20:43:52.046535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046515    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0314d62b-8300-4226-b3f2-1d77768ae56f" containerName="payload-pre-processing"
Jul 24 20:43:52.046673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046577    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0314d62b-8300-4226-b3f2-1d77768ae56f" containerName="payload-pre-processing"
Jul 24 20:43:52.046673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.046586    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7889e4-50f9-49da-8b5d-c95d36d7887b" containerName="payload-processing"
Jul 24 20:43:52.051195 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.051173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.053919 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.053896    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bd4a88b3-dockercfg-cfsb5\""
Jul 24 20:43:52.054172 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.054153    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bd4a88b3\""
Jul 24 20:43:52.057482 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.057458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:43:52.062743 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.062719    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:43:52.066807 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.066788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.074115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.074095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:43:52.150209 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.150177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4l6w\" (UniqueName: \"kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.150399 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.150220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.150399 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.150245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfc55\" (UniqueName: \"kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.150399 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.150323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.250914 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.250818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4l6w\" (UniqueName: \"kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.250914 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.250882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.250914 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.250907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfc55\" (UniqueName: \"kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.251389 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.250937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.251562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.251543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.251683 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.251665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.259627 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.259572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfc55\" (UniqueName: \"kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55\") pod \"payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.259747 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.259570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4l6w\" (UniqueName: \"kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w\") pod \"payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") " pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.363539 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.363507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.378519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.378491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.506375 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.506263    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:43:52.511140 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:43:52.511095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ac1fdf_7d15_43dd_811d_85537cd279c7.slice/crio-cac3414581a448aee768bcacc69aae5b445c162f9c29ea33f97a3bcdde426f80 WatchSource:0}: Error finding container cac3414581a448aee768bcacc69aae5b445c162f9c29ea33f97a3bcdde426f80: Status 404 returned error can't find the container with id cac3414581a448aee768bcacc69aae5b445c162f9c29ea33f97a3bcdde426f80
Jul 24 20:43:52.533805 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.533733    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:43:52.539385 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:43:52.539344    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740802d6_ae70_4cc8_81f2_41ff6aed9689.slice/crio-9d88316d6378900c6497f68a636cb00459141c6b0c4d7536e1109b36935bc3fe WatchSource:0}: Error finding container 9d88316d6378900c6497f68a636cb00459141c6b0c4d7536e1109b36935bc3fe: Status 404 returned error can't find the container with id 9d88316d6378900c6497f68a636cb00459141c6b0c4d7536e1109b36935bc3fe
Jul 24 20:43:52.933393 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.933350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" event={"ID":"740802d6-ae70-4cc8-81f2-41ff6aed9689","Type":"ContainerStarted","Data":"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"}
Jul 24 20:43:52.933625 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.933416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" event={"ID":"740802d6-ae70-4cc8-81f2-41ff6aed9689","Type":"ContainerStarted","Data":"9d88316d6378900c6497f68a636cb00459141c6b0c4d7536e1109b36935bc3fe"}
Jul 24 20:43:52.933625 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.933445    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:43:52.934867 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.934841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" event={"ID":"f4ac1fdf-7d15-43dd-811d-85537cd279c7","Type":"ContainerStarted","Data":"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"}
Jul 24 20:43:52.934946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.934890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" event={"ID":"f4ac1fdf-7d15-43dd-811d-85537cd279c7","Type":"ContainerStarted","Data":"cac3414581a448aee768bcacc69aae5b445c162f9c29ea33f97a3bcdde426f80"}
Jul 24 20:43:52.935012 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.934980    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:43:52.949894 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.949851    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" podStartSLOduration=0.949837815 podStartE2EDuration="949.837815ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:43:52.948762371 +0000 UTC m=+1699.302339538" watchObservedRunningTime="2026-07-24 20:43:52.949837815 +0000 UTC m=+1699.303414974"
Jul 24 20:43:52.963979 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:52.963940    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" podStartSLOduration=0.963927991 podStartE2EDuration="963.927991ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:43:52.96335251 +0000 UTC m=+1699.316929650" watchObservedRunningTime="2026-07-24 20:43:52.963927991 +0000 UTC m=+1699.317505179"
Jul 24 20:43:58.988681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:43:58.988643    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:44:03.942097 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:03.942064    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:44:03.942523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:03.942131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:44:27.647277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.647243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:27.679948 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.679920    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:44:27.680157 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.680123    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" podUID="740802d6-ae70-4cc8-81f2-41ff6aed9689" containerName="payload-processing" containerID="cri-o://558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4" gracePeriod=30
Jul 24 20:44:27.684649 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.683541    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:44:27.684649 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.683788    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" podUID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" containerName="payload-pre-processing" containerID="cri-o://73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65" gracePeriod=30
Jul 24 20:44:27.715965 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.715926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:27.772397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.772367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:27.957400 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.957376    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:44:27.960465 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:27.960444    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:44:28.050549 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.050512    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume\") pod \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") "
Jul 24 20:44:28.050735 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.050582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfc55\" (UniqueName: \"kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55\") pod \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\" (UID: \"f4ac1fdf-7d15-43dd-811d-85537cd279c7\") "
Jul 24 20:44:28.050784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.050734    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4l6w\" (UniqueName: \"kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w\") pod \"740802d6-ae70-4cc8-81f2-41ff6aed9689\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") "
Jul 24 20:44:28.050831 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.050810    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume\") pod \"740802d6-ae70-4cc8-81f2-41ff6aed9689\" (UID: \"740802d6-ae70-4cc8-81f2-41ff6aed9689\") "
Jul 24 20:44:28.051138 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.050990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4ac1fdf-7d15-43dd-811d-85537cd279c7" (UID: "f4ac1fdf-7d15-43dd-811d-85537cd279c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:44:28.051248 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.051167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "740802d6-ae70-4cc8-81f2-41ff6aed9689" (UID: "740802d6-ae70-4cc8-81f2-41ff6aed9689"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:44:28.052703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.052684    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55" (OuterVolumeSpecName: "kube-api-access-zfc55") pod "f4ac1fdf-7d15-43dd-811d-85537cd279c7" (UID: "f4ac1fdf-7d15-43dd-811d-85537cd279c7"). InnerVolumeSpecName "kube-api-access-zfc55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:44:28.052873 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.052842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w" (OuterVolumeSpecName: "kube-api-access-f4l6w") pod "740802d6-ae70-4cc8-81f2-41ff6aed9689" (UID: "740802d6-ae70-4cc8-81f2-41ff6aed9689"). InnerVolumeSpecName "kube-api-access-f4l6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:44:28.107091 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.107055    2577 generic.go:356] "Generic (PLEG): container finished" podID="740802d6-ae70-4cc8-81f2-41ff6aed9689" containerID="558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4" exitCode=0
Jul 24 20:44:28.107251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.107107    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"
Jul 24 20:44:28.107251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.107140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" event={"ID":"740802d6-ae70-4cc8-81f2-41ff6aed9689","Type":"ContainerDied","Data":"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"}
Jul 24 20:44:28.107251 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.107227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr" event={"ID":"740802d6-ae70-4cc8-81f2-41ff6aed9689","Type":"ContainerDied","Data":"9d88316d6378900c6497f68a636cb00459141c6b0c4d7536e1109b36935bc3fe"}
Jul 24 20:44:28.107405 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.107255    2577 scope.go:117] "RemoveContainer" containerID="558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"
Jul 24 20:44:28.108547 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.108525    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" containerID="73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65" exitCode=0
Jul 24 20:44:28.108668 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.108566    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"
Jul 24 20:44:28.108668 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.108572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" event={"ID":"f4ac1fdf-7d15-43dd-811d-85537cd279c7","Type":"ContainerDied","Data":"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"}
Jul 24 20:44:28.108668 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.108629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b" event={"ID":"f4ac1fdf-7d15-43dd-811d-85537cd279c7","Type":"ContainerDied","Data":"cac3414581a448aee768bcacc69aae5b445c162f9c29ea33f97a3bcdde426f80"}
Jul 24 20:44:28.117746 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.117551    2577 scope.go:117] "RemoveContainer" containerID="558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"
Jul 24 20:44:28.117982 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:44:28.117961    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4\": container with ID starting with 558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4 not found: ID does not exist" containerID="558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"
Jul 24 20:44:28.118067 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.117990    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4"} err="failed to get container status \"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4\": rpc error: code = NotFound desc = could not find container \"558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4\": container with ID starting with 558037ef879765580ae3ee549f099165e80c5fd600638e17c4555ee031f9eba4 not found: ID does not exist"
Jul 24 20:44:28.118067 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.118007    2577 scope.go:117] "RemoveContainer" containerID="73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"
Jul 24 20:44:28.126047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.126029    2577 scope.go:117] "RemoveContainer" containerID="73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"
Jul 24 20:44:28.126287 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:44:28.126266    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65\": container with ID starting with 73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65 not found: ID does not exist" containerID="73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"
Jul 24 20:44:28.126332 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.126293    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65"} err="failed to get container status \"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65\": rpc error: code = NotFound desc = could not find container \"73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65\": container with ID starting with 73679a3ac40ed97bae306fd8332ab7fcb786a0afd758a86954cf48e4a5634d65 not found: ID does not exist"
Jul 24 20:44:28.133174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.133150    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:44:28.137500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.137482    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bd4a88b3-c8c587d68-lbs6b"]
Jul 24 20:44:28.151997 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.151974    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ac1fdf-7d15-43dd-811d-85537cd279c7-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:44:28.151997 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.151992    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfc55\" (UniqueName: \"kubernetes.io/projected/f4ac1fdf-7d15-43dd-811d-85537cd279c7-kube-api-access-zfc55\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:44:28.152137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.152003    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4l6w\" (UniqueName: \"kubernetes.io/projected/740802d6-ae70-4cc8-81f2-41ff6aed9689-kube-api-access-f4l6w\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:44:28.152137 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.152012    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/740802d6-ae70-4cc8-81f2-41ff6aed9689-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:44:28.152855 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.152837    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" path="/var/lib/kubelet/pods/f4ac1fdf-7d15-43dd-811d-85537cd279c7/volumes"
Jul 24 20:44:28.153080 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.153067    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:44:28.153120 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:28.153093    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bd4a88b3-5489cfb6c6-dfvhr"]
Jul 24 20:44:30.151524 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:30.151490    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="740802d6-ae70-4cc8-81f2-41ff6aed9689" path="/var/lib/kubelet/pods/740802d6-ae70-4cc8-81f2-41ff6aed9689/volumes"
Jul 24 20:44:30.393919 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:30.393887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:44:32.335464 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:32.335429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:33.632746 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:33.632710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:33.891016 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:33.890931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:34.146296 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:34.146214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:34.480560 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:34.480483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:34.798259 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:34.798225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:36.823117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:36.823085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:36.901398 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:36.901372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:36.967906 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:36.967878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:37.270841 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.270808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:37.398987 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.398956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:37.487422 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.487386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:44:37.511205 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511171    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:44:37.511577 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511565    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740802d6-ae70-4cc8-81f2-41ff6aed9689" containerName="payload-processing"
Jul 24 20:44:37.511647 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511580    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="740802d6-ae70-4cc8-81f2-41ff6aed9689" containerName="payload-processing"
Jul 24 20:44:37.511647 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511609    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" containerName="payload-pre-processing"
Jul 24 20:44:37.511647 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511615    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" containerName="payload-pre-processing"
Jul 24 20:44:37.511741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511690    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="740802d6-ae70-4cc8-81f2-41ff6aed9689" containerName="payload-processing"
Jul 24 20:44:37.511741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.511700    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ac1fdf-7d15-43dd-811d-85537cd279c7" containerName="payload-pre-processing"
Jul 24 20:44:37.514627 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.514609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.517150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.517130    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2c1cd65e\""
Jul 24 20:44:37.517247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.517187    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2c1cd65e-dockercfg-w726q\""
Jul 24 20:44:37.521727 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.521665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.521862 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.521771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pljk\" (UniqueName: \"kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.531887 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.531860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:44:37.540965 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.540940    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:44:37.544849 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.544828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.553818 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.553792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:44:37.622310 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.622275    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.622486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.622323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svmp\" (UniqueName: \"kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.622486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.622352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.622486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.622455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pljk\" (UniqueName: \"kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.623131 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.623112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.631096 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.631070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pljk\" (UniqueName: \"kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk\") pod \"payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.723373 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.723337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.723544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.723386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svmp\" (UniqueName: \"kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.724061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.724040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.733254 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.733226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svmp\" (UniqueName: \"kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp\") pod \"payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") " pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.824300 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.824215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:37.855319 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.855283    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:37.977110 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:37.976989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:44:37.980107 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:44:37.980069    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d9b0b0_a9ab_4037_8317_f067def4165a.slice/crio-f0e0c196a2bddc947ee8a6c16dab2a5f2e8392421c12cf34d5e437d37d8f8904 WatchSource:0}: Error finding container f0e0c196a2bddc947ee8a6c16dab2a5f2e8392421c12cf34d5e437d37d8f8904: Status 404 returned error can't find the container with id f0e0c196a2bddc947ee8a6c16dab2a5f2e8392421c12cf34d5e437d37d8f8904
Jul 24 20:44:38.001307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.001281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:44:38.013467 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:44:38.013435    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8c85f2_6803_4fa1_b72e_9a32ea2fdbff.slice/crio-7fa4770a22f8412b55cc24862baa7040d9047ba7784fc35fe6c115c03453f649 WatchSource:0}: Error finding container 7fa4770a22f8412b55cc24862baa7040d9047ba7784fc35fe6c115c03453f649: Status 404 returned error can't find the container with id 7fa4770a22f8412b55cc24862baa7040d9047ba7784fc35fe6c115c03453f649
Jul 24 20:44:38.161240 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.161154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" event={"ID":"18d9b0b0-a9ab-4037-8317-f067def4165a","Type":"ContainerStarted","Data":"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"}
Jul 24 20:44:38.161240 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.161205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" event={"ID":"18d9b0b0-a9ab-4037-8317-f067def4165a","Type":"ContainerStarted","Data":"f0e0c196a2bddc947ee8a6c16dab2a5f2e8392421c12cf34d5e437d37d8f8904"}
Jul 24 20:44:38.161460 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.161253    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:44:38.162652 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.162630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" event={"ID":"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff","Type":"ContainerStarted","Data":"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"}
Jul 24 20:44:38.162757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.162666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" event={"ID":"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff","Type":"ContainerStarted","Data":"7fa4770a22f8412b55cc24862baa7040d9047ba7784fc35fe6c115c03453f649"}
Jul 24 20:44:38.162757 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.162711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:38.176988 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.176948    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" podStartSLOduration=1.176936499 podStartE2EDuration="1.176936499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:44:38.175193046 +0000 UTC m=+1744.528770205" watchObservedRunningTime="2026-07-24 20:44:38.176936499 +0000 UTC m=+1744.530513640"
Jul 24 20:44:38.190274 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:38.190227    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" podStartSLOduration=1.190211632 podStartE2EDuration="1.190211632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:44:38.190188227 +0000 UTC m=+1744.543765359" watchObservedRunningTime="2026-07-24 20:44:38.190211632 +0000 UTC m=+1744.543788775"
Jul 24 20:44:42.588405 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:42.588372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:44:49.169723 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:49.169691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:44:49.170171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:44:49.169870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:45:12.601102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.601070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:12.619866 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.619835    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:45:12.620144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.620104    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" podUID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" containerName="payload-processing" containerID="cri-o://bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771" gracePeriod=30
Jul 24 20:45:12.625286 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.624899    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:45:12.625286 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.625142    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" podUID="18d9b0b0-a9ab-4037-8317-f067def4165a" containerName="payload-pre-processing" containerID="cri-o://14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad" gracePeriod=30
Jul 24 20:45:12.665481 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.665453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:12.766706 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.766662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:12.897434 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.897412    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:45:12.900265 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:12.900249    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:45:13.041117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041088    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume\") pod \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") "
Jul 24 20:45:13.041290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pljk\" (UniqueName: \"kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk\") pod \"18d9b0b0-a9ab-4037-8317-f067def4165a\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") "
Jul 24 20:45:13.041290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041192    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7svmp\" (UniqueName: \"kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp\") pod \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\" (UID: \"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff\") "
Jul 24 20:45:13.041290 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume\") pod \"18d9b0b0-a9ab-4037-8317-f067def4165a\" (UID: \"18d9b0b0-a9ab-4037-8317-f067def4165a\") "
Jul 24 20:45:13.041651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" (UID: "9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:45:13.041791 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.041677    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18d9b0b0-a9ab-4037-8317-f067def4165a" (UID: "18d9b0b0-a9ab-4037-8317-f067def4165a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:45:13.043414 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.043394    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp" (OuterVolumeSpecName: "kube-api-access-7svmp") pod "9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" (UID: "9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff"). InnerVolumeSpecName "kube-api-access-7svmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:45:13.043488 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.043418    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk" (OuterVolumeSpecName: "kube-api-access-4pljk") pod "18d9b0b0-a9ab-4037-8317-f067def4165a" (UID: "18d9b0b0-a9ab-4037-8317-f067def4165a"). InnerVolumeSpecName "kube-api-access-4pljk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:45:13.142492 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.142411    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:45:13.142492 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.142442    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pljk\" (UniqueName: \"kubernetes.io/projected/18d9b0b0-a9ab-4037-8317-f067def4165a-kube-api-access-4pljk\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:45:13.142492 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.142451    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7svmp\" (UniqueName: \"kubernetes.io/projected/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff-kube-api-access-7svmp\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:45:13.142492 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.142460    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18d9b0b0-a9ab-4037-8317-f067def4165a-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:45:13.335186 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.335151    2577 generic.go:356] "Generic (PLEG): container finished" podID="18d9b0b0-a9ab-4037-8317-f067def4165a" containerID="14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad" exitCode=0
Jul 24 20:45:13.335365 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.335211    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"
Jul 24 20:45:13.335365 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.335240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" event={"ID":"18d9b0b0-a9ab-4037-8317-f067def4165a","Type":"ContainerDied","Data":"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"}
Jul 24 20:45:13.335365 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.335287    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl" event={"ID":"18d9b0b0-a9ab-4037-8317-f067def4165a","Type":"ContainerDied","Data":"f0e0c196a2bddc947ee8a6c16dab2a5f2e8392421c12cf34d5e437d37d8f8904"}
Jul 24 20:45:13.335365 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.335315    2577 scope.go:117] "RemoveContainer" containerID="14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"
Jul 24 20:45:13.336659 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.336634    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" containerID="bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771" exitCode=0
Jul 24 20:45:13.336787 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.336674    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"
Jul 24 20:45:13.336787 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.336693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" event={"ID":"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff","Type":"ContainerDied","Data":"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"}
Jul 24 20:45:13.336787 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.336728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs" event={"ID":"9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff","Type":"ContainerDied","Data":"7fa4770a22f8412b55cc24862baa7040d9047ba7784fc35fe6c115c03453f649"}
Jul 24 20:45:13.344745 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.344723    2577 scope.go:117] "RemoveContainer" containerID="14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"
Jul 24 20:45:13.345041 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:45:13.345023    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad\": container with ID starting with 14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad not found: ID does not exist" containerID="14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"
Jul 24 20:45:13.345102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.345050    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad"} err="failed to get container status \"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad\": rpc error: code = NotFound desc = could not find container \"14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad\": container with ID starting with 14f10ffe6bf61820886b26f7199d741deac14963808b392e6e27947ba24d21ad not found: ID does not exist"
Jul 24 20:45:13.345102 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.345068    2577 scope.go:117] "RemoveContainer" containerID="bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"
Jul 24 20:45:13.352968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.352953    2577 scope.go:117] "RemoveContainer" containerID="bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"
Jul 24 20:45:13.353211 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:45:13.353191    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771\": container with ID starting with bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771 not found: ID does not exist" containerID="bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"
Jul 24 20:45:13.353277 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.353219    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771"} err="failed to get container status \"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771\": rpc error: code = NotFound desc = could not find container \"bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771\": container with ID starting with bace61ef601468546b0e832ba92e994ad8d8fedea569b971602e56b76be73771 not found: ID does not exist"
Jul 24 20:45:13.358826 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.358799    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:45:13.362224 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.362202    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2c1cd65e-744dfcd494-9bsvs"]
Jul 24 20:45:13.373445 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.373420    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:45:13.375734 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:13.375716    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2c1cd65e-646b4bdf96-59rcl"]
Jul 24 20:45:14.152723 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:14.152681    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d9b0b0-a9ab-4037-8317-f067def4165a" path="/var/lib/kubelet/pods/18d9b0b0-a9ab-4037-8317-f067def4165a/volumes"
Jul 24 20:45:14.153230 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:14.153150    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" path="/var/lib/kubelet/pods/9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff/volumes"
Jul 24 20:45:15.393287 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:15.393255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:45:17.125257 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:17.125216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:17.297858 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:17.297825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:17.552362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:17.552331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:18.075844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:18.075809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:18.173905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:18.173874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:18.273847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:18.273818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:19.895968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:19.895935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.395656 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.395564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.452657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.452626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.670475 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.670391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.736206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.736178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.820848 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.820814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:25.922804 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:25.922728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:26.021368 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:26.021336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:26.400645 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:26.400585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:52.413306 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:52.413271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:58.694864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:58.694831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:58.801944 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:58.801909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:45:58.896622 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:45:58.896562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:14.869451 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:14.869412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:20.478905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:20.478872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:20.541520 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:20.541488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:20.779934 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:20.779899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:20.878987 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:20.878935    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:31.282381 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:31.282344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:32.180645 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:32.180581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:34.965355 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:34.965315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:38.571066 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:38.571035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:38.687397 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:38.687364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:38.771303 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:38.771269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:38.871211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:38.871133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:39.750447 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:39.750411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:39.846986 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:39.846952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:39.974432 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:39.974400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:41.830841 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:41.830806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:42.189562 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:42.189472    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:46:47.529707 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:47.529670    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:47.637431 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:47.637394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:48.251540 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:48.251505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:54.142164 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:54.142131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:54.204801 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:54.204767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:54.297258 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:54.297225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:46:54.371895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:54.371864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:54.678235 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:54.678202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:55.099533 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:55.099498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.240272 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.240235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.328033 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.328005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.394356 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.394319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.495479 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.495401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.902244 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.902212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.984212 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.984184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:46:58.988882 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.988855    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:46:58.989218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989206    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d9b0b0-a9ab-4037-8317-f067def4165a" containerName="payload-pre-processing"
Jul 24 20:46:58.989269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989221    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d9b0b0-a9ab-4037-8317-f067def4165a" containerName="payload-pre-processing"
Jul 24 20:46:58.989269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989232    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" containerName="payload-processing"
Jul 24 20:46:58.989269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989238    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" containerName="payload-processing"
Jul 24 20:46:58.989371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989316    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d9b0b0-a9ab-4037-8317-f067def4165a" containerName="payload-pre-processing"
Jul 24 20:46:58.989371 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.989324    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b8c85f2-6803-4fa1-b72e-9a32ea2fdbff" containerName="payload-processing"
Jul 24 20:46:58.992509 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.992489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:58.995064 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.995039    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a3b50177\""
Jul 24 20:46:58.995192 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:58.995154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a3b50177-dockercfg-l4rnt\""
Jul 24 20:46:59.002486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.002464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:46:59.027910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.027880    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:46:59.031213 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.031196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.044755 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.044729    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:46:59.104075 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.104038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.104260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.104141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt4hj\" (UniqueName: \"kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.104260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.104180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g52tr\" (UniqueName: \"kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.104260 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.104214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.204940 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.204858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.205090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.204949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt4hj\" (UniqueName: \"kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.205090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.204999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g52tr\" (UniqueName: \"kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.205090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.205041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.205825 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.205800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.205875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.205811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.214022 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.213999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt4hj\" (UniqueName: \"kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj\") pod \"payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") " pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.214144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.214005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g52tr\" (UniqueName: \"kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr\") pod \"payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.303929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.303894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.341334 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.341287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.448020 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.447989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:46:59.450327 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:46:59.450292    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c0cb57_75f1_4a92_81d6_136e3cebeab7.slice/crio-bf31b2cd05d849116969dfacd509db769b6470c035c0ae3902349e8e1659aaab WatchSource:0}: Error finding container bf31b2cd05d849116969dfacd509db769b6470c035c0ae3902349e8e1659aaab: Status 404 returned error can't find the container with id bf31b2cd05d849116969dfacd509db769b6470c035c0ae3902349e8e1659aaab
Jul 24 20:46:59.484395 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.484372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:46:59.487689 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:46:59.487661    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode508845c_65e8_4f26_a426_ebddb3a59020.slice/crio-7079d29ac3d8d18cf3781497bc9457e8f755a8c01585ae3663fea1741dfcb6b7 WatchSource:0}: Error finding container 7079d29ac3d8d18cf3781497bc9457e8f755a8c01585ae3663fea1741dfcb6b7: Status 404 returned error can't find the container with id 7079d29ac3d8d18cf3781497bc9457e8f755a8c01585ae3663fea1741dfcb6b7
Jul 24 20:46:59.832211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.832169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" event={"ID":"b8c0cb57-75f1-4a92-81d6-136e3cebeab7","Type":"ContainerStarted","Data":"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"}
Jul 24 20:46:59.832424 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.832238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" event={"ID":"b8c0cb57-75f1-4a92-81d6-136e3cebeab7","Type":"ContainerStarted","Data":"bf31b2cd05d849116969dfacd509db769b6470c035c0ae3902349e8e1659aaab"}
Jul 24 20:46:59.832424 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.832268    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:46:59.833832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.833809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" event={"ID":"e508845c-65e8-4f26-a426-ebddb3a59020","Type":"ContainerStarted","Data":"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"}
Jul 24 20:46:59.833938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.833853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" event={"ID":"e508845c-65e8-4f26-a426-ebddb3a59020","Type":"ContainerStarted","Data":"7079d29ac3d8d18cf3781497bc9457e8f755a8c01585ae3663fea1741dfcb6b7"}
Jul 24 20:46:59.833938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.833878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:46:59.846952 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.846907    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" podStartSLOduration=1.846892279 podStartE2EDuration="1.846892279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:46:59.84661968 +0000 UTC m=+1886.200196830" watchObservedRunningTime="2026-07-24 20:46:59.846892279 +0000 UTC m=+1886.200469420"
Jul 24 20:46:59.861973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:46:59.861904    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" podStartSLOduration=0.86188585 podStartE2EDuration="861.88585ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:46:59.861155116 +0000 UTC m=+1886.214732277" watchObservedRunningTime="2026-07-24 20:46:59.86188585 +0000 UTC m=+1886.215462992"
Jul 24 20:47:02.758946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:02.758908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:03.468835 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:03.468798    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:03.602632 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:03.602579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:03.964216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:03.964179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:08.390412 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:08.390366    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:47:09.730170 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:09.730125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:09.790962 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:09.790930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:10.840402 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:10.840375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:47:10.840856 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:10.840441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:47:20.515121 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.515081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:20.542637 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.542585    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:47:20.542875 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.542850    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" podUID="e508845c-65e8-4f26-a426-ebddb3a59020" containerName="payload-processing" containerID="cri-o://bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695" gracePeriod=30
Jul 24 20:47:20.548663 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.548567    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:47:20.548910 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.548836    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" podUID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" containerName="payload-pre-processing" containerID="cri-o://76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23" gracePeriod=30
Jul 24 20:47:20.581944 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.581918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:20.641199 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.641163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:20.824216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.824194    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:47:20.827130 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.827109    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:47:20.906806 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.906771    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume\") pod \"e508845c-65e8-4f26-a426-ebddb3a59020\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") "
Jul 24 20:47:20.906806 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.906808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt4hj\" (UniqueName: \"kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj\") pod \"e508845c-65e8-4f26-a426-ebddb3a59020\" (UID: \"e508845c-65e8-4f26-a426-ebddb3a59020\") "
Jul 24 20:47:20.907051 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.906825    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g52tr\" (UniqueName: \"kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr\") pod \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") "
Jul 24 20:47:20.907051 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.906848    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume\") pod \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\" (UID: \"b8c0cb57-75f1-4a92-81d6-136e3cebeab7\") "
Jul 24 20:47:20.907269 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.907241    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8c0cb57-75f1-4a92-81d6-136e3cebeab7" (UID: "b8c0cb57-75f1-4a92-81d6-136e3cebeab7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:47:20.907384 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.907244    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e508845c-65e8-4f26-a426-ebddb3a59020" (UID: "e508845c-65e8-4f26-a426-ebddb3a59020"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:47:20.909067 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.909049    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj" (OuterVolumeSpecName: "kube-api-access-vt4hj") pod "e508845c-65e8-4f26-a426-ebddb3a59020" (UID: "e508845c-65e8-4f26-a426-ebddb3a59020"). InnerVolumeSpecName "kube-api-access-vt4hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:47:20.909140 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.909125    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr" (OuterVolumeSpecName: "kube-api-access-g52tr") pod "b8c0cb57-75f1-4a92-81d6-136e3cebeab7" (UID: "b8c0cb57-75f1-4a92-81d6-136e3cebeab7"). InnerVolumeSpecName "kube-api-access-g52tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:47:20.939673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.939643    2577 generic.go:356] "Generic (PLEG): container finished" podID="e508845c-65e8-4f26-a426-ebddb3a59020" containerID="bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695" exitCode=0
Jul 24 20:47:20.939807 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.939685    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"
Jul 24 20:47:20.939807 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.939732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" event={"ID":"e508845c-65e8-4f26-a426-ebddb3a59020","Type":"ContainerDied","Data":"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"}
Jul 24 20:47:20.939807 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.939790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx" event={"ID":"e508845c-65e8-4f26-a426-ebddb3a59020","Type":"ContainerDied","Data":"7079d29ac3d8d18cf3781497bc9457e8f755a8c01585ae3663fea1741dfcb6b7"}
Jul 24 20:47:20.939947 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.939824    2577 scope.go:117] "RemoveContainer" containerID="bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"
Jul 24 20:47:20.941079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.941058    2577 generic.go:356] "Generic (PLEG): container finished" podID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" containerID="76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23" exitCode=0
Jul 24 20:47:20.941197 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.941100    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"
Jul 24 20:47:20.941197 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.941147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" event={"ID":"b8c0cb57-75f1-4a92-81d6-136e3cebeab7","Type":"ContainerDied","Data":"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"}
Jul 24 20:47:20.941298 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.941201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc" event={"ID":"b8c0cb57-75f1-4a92-81d6-136e3cebeab7","Type":"ContainerDied","Data":"bf31b2cd05d849116969dfacd509db769b6470c035c0ae3902349e8e1659aaab"}
Jul 24 20:47:20.948693 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.948543    2577 scope.go:117] "RemoveContainer" containerID="bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"
Jul 24 20:47:20.948935 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:47:20.948909    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695\": container with ID starting with bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695 not found: ID does not exist" containerID="bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"
Jul 24 20:47:20.949041 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.948944    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695"} err="failed to get container status \"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695\": rpc error: code = NotFound desc = could not find container \"bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695\": container with ID starting with bcd108a65ac8627cff6c552a8d3e0eb9eac6f782e47ce45be90853e95579b695 not found: ID does not exist"
Jul 24 20:47:20.949041 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.948967    2577 scope.go:117] "RemoveContainer" containerID="76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"
Jul 24 20:47:20.956719 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.956701    2577 scope.go:117] "RemoveContainer" containerID="76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"
Jul 24 20:47:20.956973 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:47:20.956956    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23\": container with ID starting with 76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23 not found: ID does not exist" containerID="76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"
Jul 24 20:47:20.957031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.956981    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23"} err="failed to get container status \"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23\": rpc error: code = NotFound desc = could not find container \"76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23\": container with ID starting with 76472607706b835ace9bfeca5fc70438cd52f1e188c4d9876f7c06323b467c23 not found: ID does not exist"
Jul 24 20:47:20.964445 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.964424    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:47:20.968920 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.968900    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a3b50177-b798b56fd-d2hdx"]
Jul 24 20:47:20.979173 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.979153    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:47:20.983143 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:20.983120    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a3b50177-7d8cb9464-2nmzc"]
Jul 24 20:47:21.008462 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:21.008441    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508845c-65e8-4f26-a426-ebddb3a59020-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:47:21.008568 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:21.008467    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt4hj\" (UniqueName: \"kubernetes.io/projected/e508845c-65e8-4f26-a426-ebddb3a59020-kube-api-access-vt4hj\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:47:21.008568 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:21.008482    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g52tr\" (UniqueName: \"kubernetes.io/projected/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-kube-api-access-g52tr\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:47:21.008568 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:21.008492    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8c0cb57-75f1-4a92-81d6-136e3cebeab7-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:47:22.151918 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:22.151887    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" path="/var/lib/kubelet/pods/b8c0cb57-75f1-4a92-81d6-136e3cebeab7/volumes"
Jul 24 20:47:22.152276 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:22.152221    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e508845c-65e8-4f26-a426-ebddb3a59020" path="/var/lib/kubelet/pods/e508845c-65e8-4f26-a426-ebddb3a59020/volumes"
Jul 24 20:47:23.089245 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:23.089209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:47:25.210841 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:25.210807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:25.638981 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:25.638945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:31.396193 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:31.396159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:31.497123 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:31.497089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:31.694075 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:31.693962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:31.936679 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:31.936637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:32.079550 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:32.079506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:32.178929 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:32.178898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:32.308067 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:32.308041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:38.040144 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:38.040111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:38.144555 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:38.144502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:38.240541 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:38.240506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:38.392112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:38.392005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:43.963295 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:43.963256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:44.064178 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:44.064128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:50.889651 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:50.889568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:51.224793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:51.224704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:51.571832 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:51.571801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.210772 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.210727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.295732 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.295699    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.366669 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.366638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.612496 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.612447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.767522 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.767486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.871247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.871164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:47:54.917155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917119    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:47:54.917489 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917478    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" containerName="payload-pre-processing"
Jul 24 20:47:54.917538 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917491    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" containerName="payload-pre-processing"
Jul 24 20:47:54.917538 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917509    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e508845c-65e8-4f26-a426-ebddb3a59020" containerName="payload-processing"
Jul 24 20:47:54.917538 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917515    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e508845c-65e8-4f26-a426-ebddb3a59020" containerName="payload-processing"
Jul 24 20:47:54.917650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e508845c-65e8-4f26-a426-ebddb3a59020" containerName="payload-processing"
Jul 24 20:47:54.917650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.917588    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c0cb57-75f1-4a92-81d6-136e3cebeab7" containerName="payload-pre-processing"
Jul 24 20:47:54.920694 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.920678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:54.923434 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.923399    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-760f3539\""
Jul 24 20:47:54.923569 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.923439    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-760f3539-dockercfg-cl4lb\""
Jul 24 20:47:54.929464 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:54.929434    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:47:55.003606 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.003571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.003764 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.003634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg5c4\" (UniqueName: \"kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.104430 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.104396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.104650 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.104439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg5c4\" (UniqueName: \"kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.105191 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.105170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.114724 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.114701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg5c4\" (UniqueName: \"kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4\") pod \"payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") " pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.232481 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.232383    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:55.363619 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:55.363573    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:47:55.365967 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:47:55.365940    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38ea122_be87_4394_bdca_18192f5eeaea.slice/crio-7331d8509ae78688e068f1eca549942067100f9902d49532ec5889a691affc97 WatchSource:0}: Error finding container 7331d8509ae78688e068f1eca549942067100f9902d49532ec5889a691affc97: Status 404 returned error can't find the container with id 7331d8509ae78688e068f1eca549942067100f9902d49532ec5889a691affc97
Jul 24 20:47:56.105535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:56.105506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" event={"ID":"a38ea122-be87-4394-bdca-18192f5eeaea","Type":"ContainerStarted","Data":"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"}
Jul 24 20:47:56.105755 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:56.105554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" event={"ID":"a38ea122-be87-4394-bdca-18192f5eeaea","Type":"ContainerStarted","Data":"7331d8509ae78688e068f1eca549942067100f9902d49532ec5889a691affc97"}
Jul 24 20:47:56.105755 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:56.105631    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:47:56.121972 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:56.121920    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" podStartSLOduration=2.12190551 podStartE2EDuration="2.12190551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:47:56.120445425 +0000 UTC m=+1942.474022580" watchObservedRunningTime="2026-07-24 20:47:56.12190551 +0000 UTC m=+1942.475482650"
Jul 24 20:47:58.760765 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:47:58.760733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:00.689490 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:00.689443    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:48:04.894207 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.894131    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:48:04.897503 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.897488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.901164 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.901140    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-760f3539\"/\"kube-root-ca.crt\""
Jul 24 20:48:04.901299 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.901164    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-760f3539\"/\"default-dockercfg-wxvwl\""
Jul 24 20:48:04.901299 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.901179    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-760f3539\"/\"openshift-service-ca.crt\""
Jul 24 20:48:04.901299 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.901140    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-760f3539\"/\"e2e-lifecycle-model-760f3539-kserve-self-signed-certs\""
Jul 24 20:48:04.907132 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.907109    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:48:04.909880 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.909856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:04.994123 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.994250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.994250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgp6c\" (UniqueName: \"kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.994250 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.994425 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:04.994425 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:04.994334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgp6c\" (UniqueName: \"kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.095970 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.096079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.096079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.095977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.097897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.097876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.098093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.098080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.104228 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.104208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgp6c\" (UniqueName: \"kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c\") pod \"e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") " pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.208245 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.208167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:05.343401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:05.343376    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:48:05.346068 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:48:05.346041    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e1124cd_3787_4e56_9cf2_a4b3e49c1394.slice/crio-6a21656cdf4ccae35c8b796ae6db16ca5cb4f1c26c551cff722c18519ec22f86 WatchSource:0}: Error finding container 6a21656cdf4ccae35c8b796ae6db16ca5cb4f1c26c551cff722c18519ec22f86: Status 404 returned error can't find the container with id 6a21656cdf4ccae35c8b796ae6db16ca5cb4f1c26c551cff722c18519ec22f86
Jul 24 20:48:06.155843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:06.155809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" event={"ID":"8e1124cd-3787-4e56-9cf2-a4b3e49c1394","Type":"ContainerStarted","Data":"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"}
Jul 24 20:48:06.156206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:06.155855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" event={"ID":"8e1124cd-3787-4e56-9cf2-a4b3e49c1394","Type":"ContainerStarted","Data":"6a21656cdf4ccae35c8b796ae6db16ca5cb4f1c26c551cff722c18519ec22f86"}
Jul 24 20:48:06.175527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:06.175447    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" podStartSLOduration=2.17542185 podStartE2EDuration="2.17542185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:48:06.171826202 +0000 UTC m=+1952.525403360" watchObservedRunningTime="2026-07-24 20:48:06.17542185 +0000 UTC m=+1952.528999013"
Jul 24 20:48:07.111994 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:07.111957    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:48:09.007218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:09.007184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:48:15.208823 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:15.208790    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:15.209261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:15.208858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:15.221262 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:15.221238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:16.212711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:16.212683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:48:23.586816 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:23.586778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:48:41.793864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:41.793784    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:48:43.707952 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.707918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:43.728029 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.727996    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:48:43.728259 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.728232    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" podUID="a38ea122-be87-4394-bdca-18192f5eeaea" containerName="payload-processing" containerID="cri-o://3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc" gracePeriod=30
Jul 24 20:48:43.805512 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.805479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:43.858464 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.858408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:43.973615 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:43.973572    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:48:44.056857 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.056830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg5c4\" (UniqueName: \"kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4\") pod \"a38ea122-be87-4394-bdca-18192f5eeaea\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") "
Jul 24 20:48:44.057025 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.056911    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume\") pod \"a38ea122-be87-4394-bdca-18192f5eeaea\" (UID: \"a38ea122-be87-4394-bdca-18192f5eeaea\") "
Jul 24 20:48:44.059487 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.057460    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a38ea122-be87-4394-bdca-18192f5eeaea" (UID: "a38ea122-be87-4394-bdca-18192f5eeaea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:48:44.065112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.065085    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4" (OuterVolumeSpecName: "kube-api-access-sg5c4") pod "a38ea122-be87-4394-bdca-18192f5eeaea" (UID: "a38ea122-be87-4394-bdca-18192f5eeaea"). InnerVolumeSpecName "kube-api-access-sg5c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:48:44.158478 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.158451    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a38ea122-be87-4394-bdca-18192f5eeaea-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:48:44.158478 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.158481    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg5c4\" (UniqueName: \"kubernetes.io/projected/a38ea122-be87-4394-bdca-18192f5eeaea-kube-api-access-sg5c4\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:48:44.338127 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.338089    2577 generic.go:356] "Generic (PLEG): container finished" podID="a38ea122-be87-4394-bdca-18192f5eeaea" containerID="3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc" exitCode=0
Jul 24 20:48:44.338307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.338161    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"
Jul 24 20:48:44.338307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.338169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" event={"ID":"a38ea122-be87-4394-bdca-18192f5eeaea","Type":"ContainerDied","Data":"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"}
Jul 24 20:48:44.338307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.338221    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk" event={"ID":"a38ea122-be87-4394-bdca-18192f5eeaea","Type":"ContainerDied","Data":"7331d8509ae78688e068f1eca549942067100f9902d49532ec5889a691affc97"}
Jul 24 20:48:44.338307 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.338242    2577 scope.go:117] "RemoveContainer" containerID="3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"
Jul 24 20:48:44.347264 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.347244    2577 scope.go:117] "RemoveContainer" containerID="3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"
Jul 24 20:48:44.347545 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:48:44.347518    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc\": container with ID starting with 3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc not found: ID does not exist" containerID="3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"
Jul 24 20:48:44.347677 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.347553    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc"} err="failed to get container status \"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc\": rpc error: code = NotFound desc = could not find container \"3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc\": container with ID starting with 3a3baba138768cc8b409d9d19745cb30231729d3c81003e3c3764e0535a51dbc not found: ID does not exist"
Jul 24 20:48:44.356166 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.356143    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:48:44.360544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:44.360524    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-760f3539-55cdc44df8-4c8zk"]
Jul 24 20:48:46.152889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:46.152851    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38ea122-be87-4394-bdca-18192f5eeaea" path="/var/lib/kubelet/pods/a38ea122-be87-4394-bdca-18192f5eeaea/volumes"
Jul 24 20:48:48.332981 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:48.332953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:51.824851 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:51.824822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:48:52.395626 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:52.395563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:48:56.808567 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:56.808524    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:48:56.904095 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:48:56.904062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:11.828437 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:11.828397    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" podUID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" containerName="main" containerID="cri-o://1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d" gracePeriod=45
Jul 24 20:49:12.098377 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.098352    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:49:12.206401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206407    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgp6c\" (UniqueName: \"kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206654 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206446    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206654 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206467    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206654 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206515    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206654 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs\") pod \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\" (UID: \"8e1124cd-3787-4e56-9cf2-a4b3e49c1394\") "
Jul 24 20:49:12.206870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:49:12.206870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home" (OuterVolumeSpecName: "home") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:49:12.206870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206822    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:49:12.206870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.206851    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.208822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.208795    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c" (OuterVolumeSpecName: "kube-api-access-xgp6c") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "kube-api-access-xgp6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:49:12.208822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.208815    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm" (OuterVolumeSpecName: "dshm") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:49:12.208969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.208873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e1124cd-3787-4e56-9cf2-a4b3e49c1394" (UID: "8e1124cd-3787-4e56-9cf2-a4b3e49c1394"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:49:12.307672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.307642    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.307672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.307665    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.307672 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.307675    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgp6c\" (UniqueName: \"kubernetes.io/projected/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-kube-api-access-xgp6c\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.307889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.307684    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.307889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.307692    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e1124cd-3787-4e56-9cf2-a4b3e49c1394-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:49:12.469845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.469754    2577 generic.go:356] "Generic (PLEG): container finished" podID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" containerID="1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d" exitCode=0
Jul 24 20:49:12.469845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.469804    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"
Jul 24 20:49:12.469845 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.469832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" event={"ID":"8e1124cd-3787-4e56-9cf2-a4b3e49c1394","Type":"ContainerDied","Data":"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"}
Jul 24 20:49:12.470131 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.469878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879" event={"ID":"8e1124cd-3787-4e56-9cf2-a4b3e49c1394","Type":"ContainerDied","Data":"6a21656cdf4ccae35c8b796ae6db16ca5cb4f1c26c551cff722c18519ec22f86"}
Jul 24 20:49:12.470131 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.469896    2577 scope.go:117] "RemoveContainer" containerID="1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"
Jul 24 20:49:12.479322 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.479306    2577 scope.go:117] "RemoveContainer" containerID="1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"
Jul 24 20:49:12.479584 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:49:12.479564    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d\": container with ID starting with 1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d not found: ID does not exist" containerID="1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"
Jul 24 20:49:12.479657 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.479612    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d"} err="failed to get container status \"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d\": rpc error: code = NotFound desc = could not find container \"1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d\": container with ID starting with 1ba39e8d0af7de2e7b7e0f4c40e31b24c9b2ebea0d3100d31b914a07b874bf3d not found: ID does not exist"
Jul 24 20:49:12.498847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.491470    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:49:12.498847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:12.495937    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-760f3539/e2e-lifecycle-model-760f3539-kserve-7986c9d767-j8879"]
Jul 24 20:49:14.152550 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:14.152513    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" path="/var/lib/kubelet/pods/8e1124cd-3787-4e56-9cf2-a4b3e49c1394/volumes"
Jul 24 20:49:33.176038 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:33.175992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:33.932937 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:33.932887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:39.446848 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:39.446812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:45.196016 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:45.195979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:45.255824 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:45.255791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:45.467993 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:45.467913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:45.756268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:45.756191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:46.891360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:46.891321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:46.991809 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:46.991768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:47.092694 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:47.092657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:47.858143 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:47.858104    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:47.959408 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:47.959372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:48.057897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:48.057858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:49.192902 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:49.192863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:49:49.609793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:49.609755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:49.710740 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:49.710714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:55.328730 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:55.328652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:55.398715 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:55.398681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:56.023763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:56.023726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:56.697856 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:56.697822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:49:58.695310 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:49:58.695276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:50:02.847921 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:02.847889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:02.947950 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:02.947920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:03.045637 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:03.045588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:13.628296 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:13.628260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:14.018382 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:14.018347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:35.086411 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:35.086379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:41.200421 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:41.200385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:41.301033 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:41.300996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:41.571540 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:41.571505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:41.894407 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:41.894315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:42.491777 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:42.491746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:45.203724 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:45.203690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:45.304870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:45.304836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:45.446220 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:45.446193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:45.683505 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:45.683470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:45.911621 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:45.911566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:46.012099 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.012066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:46.019136 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019111    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:50:46.019472 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019461    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38ea122-be87-4394-bdca-18192f5eeaea" containerName="payload-processing"
Jul 24 20:50:46.019518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019474    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38ea122-be87-4394-bdca-18192f5eeaea" containerName="payload-processing"
Jul 24 20:50:46.019518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019490    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" containerName="main"
Jul 24 20:50:46.019518 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019496    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" containerName="main"
Jul 24 20:50:46.019679 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019558    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e1124cd-3787-4e56-9cf2-a4b3e49c1394" containerName="main"
Jul 24 20:50:46.019679 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.019567    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38ea122-be87-4394-bdca-18192f5eeaea" containerName="payload-processing"
Jul 24 20:50:46.021918 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.021903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.024811 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.024796    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c97e06\""
Jul 24 20:50:46.024927 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.024877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c97e06-dockercfg-rjnrw\""
Jul 24 20:50:46.034286 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.034264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:50:46.037450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.037432    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:50:46.040822 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.040806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.049574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.049548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.050561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.049877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl52f\" (UniqueName: \"kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.051350 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.051328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:50:46.156879 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.156846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.157049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.156942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wj7b\" (UniqueName: \"kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.157049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.157009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl52f\" (UniqueName: \"kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.157049 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.157036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.157649 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.157624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.165360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.165331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl52f\" (UniqueName: \"kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f\") pod \"payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.258207 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.258173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wj7b\" (UniqueName: \"kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.258711 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.258224    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.258904 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.258885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.266193 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.266142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wj7b\" (UniqueName: \"kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b\") pod \"payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.333192 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.333168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.352515 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.352484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.480632 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.480581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:50:46.483738 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:50:46.483705    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a31f94_2b1f_4ab9_b470_a7d9c793747d.slice/crio-5506b2b7eb672c99469e8200c3b72fd166d48c38a685ac503b7969805f6b2632 WatchSource:0}: Error finding container 5506b2b7eb672c99469e8200c3b72fd166d48c38a685ac503b7969805f6b2632: Status 404 returned error can't find the container with id 5506b2b7eb672c99469e8200c3b72fd166d48c38a685ac503b7969805f6b2632
Jul 24 20:50:46.505810 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.505789    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:50:46.519839 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:50:46.519813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8306ef7_bb5a_4cf7_9333_3217e82c75e8.slice/crio-3f7bc79664cc8a6f412715b284139ea5c39a2afbc51cadad566f6f4ddc54a024 WatchSource:0}: Error finding container 3f7bc79664cc8a6f412715b284139ea5c39a2afbc51cadad566f6f4ddc54a024: Status 404 returned error can't find the container with id 3f7bc79664cc8a6f412715b284139ea5c39a2afbc51cadad566f6f4ddc54a024
Jul 24 20:50:46.901505 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.901411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" event={"ID":"c8306ef7-bb5a-4cf7-9333-3217e82c75e8","Type":"ContainerStarted","Data":"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"}
Jul 24 20:50:46.901505 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.901465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" event={"ID":"c8306ef7-bb5a-4cf7-9333-3217e82c75e8","Type":"ContainerStarted","Data":"3f7bc79664cc8a6f412715b284139ea5c39a2afbc51cadad566f6f4ddc54a024"}
Jul 24 20:50:46.901784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.901524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:50:46.902895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.902871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" event={"ID":"f7a31f94-2b1f-4ab9-b470-a7d9c793747d","Type":"ContainerStarted","Data":"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"}
Jul 24 20:50:46.902994 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.902915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" event={"ID":"f7a31f94-2b1f-4ab9-b470-a7d9c793747d","Type":"ContainerStarted","Data":"5506b2b7eb672c99469e8200c3b72fd166d48c38a685ac503b7969805f6b2632"}
Jul 24 20:50:46.902994 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.902943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:46.934922 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.934876    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" podStartSLOduration=0.934864988 podStartE2EDuration="934.864988ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:50:46.933065268 +0000 UTC m=+2113.286642429" watchObservedRunningTime="2026-07-24 20:50:46.934864988 +0000 UTC m=+2113.288442128"
Jul 24 20:50:46.964537 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:46.964493    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" podStartSLOduration=0.964482127 podStartE2EDuration="964.482127ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:50:46.962354246 +0000 UTC m=+2113.315931384" watchObservedRunningTime="2026-07-24 20:50:46.964482127 +0000 UTC m=+2113.318059266"
Jul 24 20:50:49.704242 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:49.704211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:50.078861 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:50.078826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:50.391641 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:50.391532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:50.488306 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:50.488260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:50.698213 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:50.698136    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:50:53.188081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:53.188048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:53.272529 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:53.272500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:53.339506 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:53.339472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:53.498105 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:53.498020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:53.974950 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:53.974920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:54.072556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.072513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:54.101349 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.101321    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:50:54.105018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.104993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.107803 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.107772    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-55d9b1\""
Jul 24 20:50:54.116325 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.116300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:50:54.231437 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.231342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vch8\" (UniqueName: \"kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.231437 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.231391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.332859 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.332821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vch8\" (UniqueName: \"kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.333044 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.332875    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.333763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.333740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.341899 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.341873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vch8\" (UniqueName: \"kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8\") pod \"payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.417418 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.417394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.756527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.756501    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:50:54.758924 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:50:54.758893    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf417a223_fd92_4b98_be01_adb805a34f88.slice/crio-9bf6f311cc06a1cd3840f279d9e837b1320c162b345c9d451403736dfd3a4d4e WatchSource:0}: Error finding container 9bf6f311cc06a1cd3840f279d9e837b1320c162b345c9d451403736dfd3a4d4e: Status 404 returned error can't find the container with id 9bf6f311cc06a1cd3840f279d9e837b1320c162b345c9d451403736dfd3a4d4e
Jul 24 20:50:54.939947 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.939913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" event={"ID":"f417a223-fd92-4b98-be01-adb805a34f88","Type":"ContainerStarted","Data":"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"}
Jul 24 20:50:54.940112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.939964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" event={"ID":"f417a223-fd92-4b98-be01-adb805a34f88","Type":"ContainerStarted","Data":"9bf6f311cc06a1cd3840f279d9e837b1320c162b345c9d451403736dfd3a4d4e"}
Jul 24 20:50:54.940112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.939994    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:50:54.956644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:54.956580    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" podStartSLOduration=0.956565197 podStartE2EDuration="956.565197ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:50:54.954448684 +0000 UTC m=+2121.308025843" watchObservedRunningTime="2026-07-24 20:50:54.956565197 +0000 UTC m=+2121.310142337"
Jul 24 20:50:57.725329 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:57.725273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:50:57.909783 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:57.909751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:50:57.910037 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:50:57.910016    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:51:04.188956 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.188924    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:51:04.192675 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.192654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.196191 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.196168    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c97e06\"/\"test-model-c97e06-kserve-self-signed-certs\""
Jul 24 20:51:04.196302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.196247    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c97e06\"/\"default-dockercfg-tdlwh\""
Jul 24 20:51:04.196302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.196261    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c97e06\"/\"kube-root-ca.crt\""
Jul 24 20:51:04.196302 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.196253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c97e06\"/\"openshift-service-ca.crt\""
Jul 24 20:51:04.200843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.200820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:51:04.222692 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.222652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:51:04.326733 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.326901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.326901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.326901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.326901 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.327035 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.326915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5snc\" (UniqueName: \"kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.427670 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.427817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.427817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.427817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5snc\" (UniqueName: \"kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.427817 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.428025 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.427822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.428078 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.428025    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.428155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.428136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.428214 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.428198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.430141 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.430119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.430314 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.430297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.435564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.435546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5snc\" (UniqueName: \"kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc\") pod \"test-model-c97e06-kserve-9bc96d757-4g2tw\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") " pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.502746 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.502674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:04.636130 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.636108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:51:04.638407 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:51:04.638375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf10203bd_a6cb_4be2_9a50_15e73a137351.slice/crio-954934fc30c2675c8217d4656fa37790672d7c3304b01b22cf58698ee2413238 WatchSource:0}: Error finding container 954934fc30c2675c8217d4656fa37790672d7c3304b01b22cf58698ee2413238: Status 404 returned error can't find the container with id 954934fc30c2675c8217d4656fa37790672d7c3304b01b22cf58698ee2413238
Jul 24 20:51:04.991854 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.991817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" event={"ID":"f10203bd-a6cb-4be2-9a50-15e73a137351","Type":"ContainerStarted","Data":"97dff65955c0c06835873a0d4a739ea4e6adf30923898b018e5cfc35651f6497"}
Jul 24 20:51:04.992028 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:04.991860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" event={"ID":"f10203bd-a6cb-4be2-9a50-15e73a137351","Type":"ContainerStarted","Data":"954934fc30c2675c8217d4656fa37790672d7c3304b01b22cf58698ee2413238"}
Jul 24 20:51:05.008632 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:05.008572    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" podStartSLOduration=1.008558551 podStartE2EDuration="1.008558551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:51:05.006656467 +0000 UTC m=+2131.360233607" watchObservedRunningTime="2026-07-24 20:51:05.008558551 +0000 UTC m=+2131.362135690"
Jul 24 20:51:05.946546 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:05.946520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:51:06.896543 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:06.896506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:51:14.503557 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:14.503524    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:14.503950 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:14.503572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:14.516357 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:14.516332    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:15.053729 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:15.053700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:51:20.498440 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:20.498407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:51:24.110494 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:24.110463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:51:32.096319 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:51:32.096236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:52:07.228909 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:07.228875    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:52:07.253403 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:07.253376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:08.190898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:08.190861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:11.497203 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:11.497172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:52:18.678786 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.678743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:18.703216 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.703187    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:52:18.703427 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.703396    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" podUID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" containerName="payload-processing" containerID="cri-o://901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644" gracePeriod=30
Jul 24 20:52:18.711970 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.711796    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:52:18.712117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.712030    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" podUID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" containerName="payload-pre-processing" containerID="cri-o://7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3" gracePeriod=30
Jul 24 20:52:18.740763 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.740732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:18.801255 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.801229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:18.980736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.980715    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:52:18.983775 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:18.983742    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:52:19.086314 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wj7b\" (UniqueName: \"kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b\") pod \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") "
Jul 24 20:52:19.086314 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume\") pod \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\" (UID: \"c8306ef7-bb5a-4cf7-9333-3217e82c75e8\") "
Jul 24 20:52:19.086519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gl52f\" (UniqueName: \"kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f\") pod \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") "
Jul 24 20:52:19.086519 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086393    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume\") pod \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\" (UID: \"f7a31f94-2b1f-4ab9-b470-a7d9c793747d\") "
Jul 24 20:52:19.086837 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086809    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8306ef7-bb5a-4cf7-9333-3217e82c75e8" (UID: "c8306ef7-bb5a-4cf7-9333-3217e82c75e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:52:19.086963 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.086849    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7a31f94-2b1f-4ab9-b470-a7d9c793747d" (UID: "f7a31f94-2b1f-4ab9-b470-a7d9c793747d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:52:19.088703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.088681    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b" (OuterVolumeSpecName: "kube-api-access-5wj7b") pod "c8306ef7-bb5a-4cf7-9333-3217e82c75e8" (UID: "c8306ef7-bb5a-4cf7-9333-3217e82c75e8"). InnerVolumeSpecName "kube-api-access-5wj7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:52:19.088803 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.088743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f" (OuterVolumeSpecName: "kube-api-access-gl52f") pod "f7a31f94-2b1f-4ab9-b470-a7d9c793747d" (UID: "f7a31f94-2b1f-4ab9-b470-a7d9c793747d"). InnerVolumeSpecName "kube-api-access-gl52f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:52:19.188660 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.188631    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:19.188660 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.188660    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gl52f\" (UniqueName: \"kubernetes.io/projected/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-kube-api-access-gl52f\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:19.188835 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.188673    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7a31f94-2b1f-4ab9-b470-a7d9c793747d-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:19.188835 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.188685    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wj7b\" (UniqueName: \"kubernetes.io/projected/c8306ef7-bb5a-4cf7-9333-3217e82c75e8-kube-api-access-5wj7b\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:19.365150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.365115    2577 generic.go:356] "Generic (PLEG): container finished" podID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" containerID="901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644" exitCode=0
Jul 24 20:52:19.365291 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.365156    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"
Jul 24 20:52:19.365291 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.365203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" event={"ID":"c8306ef7-bb5a-4cf7-9333-3217e82c75e8","Type":"ContainerDied","Data":"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"}
Jul 24 20:52:19.365291 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.365243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md" event={"ID":"c8306ef7-bb5a-4cf7-9333-3217e82c75e8","Type":"ContainerDied","Data":"3f7bc79664cc8a6f412715b284139ea5c39a2afbc51cadad566f6f4ddc54a024"}
Jul 24 20:52:19.365291 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.365264    2577 scope.go:117] "RemoveContainer" containerID="901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"
Jul 24 20:52:19.366404 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.366385    2577 generic.go:356] "Generic (PLEG): container finished" podID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" containerID="7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3" exitCode=0
Jul 24 20:52:19.366486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.366431    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"
Jul 24 20:52:19.366486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.366443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" event={"ID":"f7a31f94-2b1f-4ab9-b470-a7d9c793747d","Type":"ContainerDied","Data":"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"}
Jul 24 20:52:19.366486 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.366472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2" event={"ID":"f7a31f94-2b1f-4ab9-b470-a7d9c793747d","Type":"ContainerDied","Data":"5506b2b7eb672c99469e8200c3b72fd166d48c38a685ac503b7969805f6b2632"}
Jul 24 20:52:19.373963 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.373946    2577 scope.go:117] "RemoveContainer" containerID="901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"
Jul 24 20:52:19.374244 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:52:19.374218    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644\": container with ID starting with 901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644 not found: ID does not exist" containerID="901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"
Jul 24 20:52:19.374345 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.374254    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644"} err="failed to get container status \"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644\": rpc error: code = NotFound desc = could not find container \"901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644\": container with ID starting with 901e39f0da891edf5ab031b335315246eee0b9f3883ae06b915549cce4749644 not found: ID does not exist"
Jul 24 20:52:19.374345 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.374281    2577 scope.go:117] "RemoveContainer" containerID="7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"
Jul 24 20:52:19.382581 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.382556    2577 scope.go:117] "RemoveContainer" containerID="7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"
Jul 24 20:52:19.382837 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:52:19.382821    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3\": container with ID starting with 7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3 not found: ID does not exist" containerID="7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"
Jul 24 20:52:19.382897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.382845    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3"} err="failed to get container status \"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3\": rpc error: code = NotFound desc = could not find container \"7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3\": container with ID starting with 7bb61b0101409e76be927da4b59772dd6eceffaf91ed45a8734dcdb629ad54d3 not found: ID does not exist"
Jul 24 20:52:19.391069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.391047    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:52:19.393351 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.393332    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c97e06-5bdbf5bcf8-pb7md"]
Jul 24 20:52:19.404224 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.404196    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:52:19.405721 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:19.405703    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c97e06-697dfc8dd4-c26s2"]
Jul 24 20:52:20.152770 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:20.152744    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" path="/var/lib/kubelet/pods/c8306ef7-bb5a-4cf7-9333-3217e82c75e8/volumes"
Jul 24 20:52:20.153151 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:20.153066    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" path="/var/lib/kubelet/pods/f7a31f94-2b1f-4ab9-b470-a7d9c793747d/volumes"
Jul 24 20:52:22.249305 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.249262    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" podUID="f10203bd-a6cb-4be2-9a50-15e73a137351" containerName="main" containerID="cri-o://97dff65955c0c06835873a0d4a739ea4e6adf30923898b018e5cfc35651f6497" gracePeriod=45
Jul 24 20:52:22.384969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.384937    2577 generic.go:356] "Generic (PLEG): container finished" podID="f10203bd-a6cb-4be2-9a50-15e73a137351" containerID="97dff65955c0c06835873a0d4a739ea4e6adf30923898b018e5cfc35651f6497" exitCode=0
Jul 24 20:52:22.384969 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.384980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" event={"ID":"f10203bd-a6cb-4be2-9a50-15e73a137351","Type":"ContainerDied","Data":"97dff65955c0c06835873a0d4a739ea4e6adf30923898b018e5cfc35651f6497"}
Jul 24 20:52:22.502750 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.502686    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:52:22.620022 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.619989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620022 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620020    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620042    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5snc\" (UniqueName: \"kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620261 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm\") pod \"f10203bd-a6cb-4be2-9a50-15e73a137351\" (UID: \"f10203bd-a6cb-4be2-9a50-15e73a137351\") "
Jul 24 20:52:22.620522 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620340    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home" (OuterVolumeSpecName: "home") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:52:22.620522 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache" (OuterVolumeSpecName: "model-cache") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:52:22.620522 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620346    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:52:22.620522 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620520    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:22.620701 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620534    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:22.620701 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.620543    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:22.622263 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.622241    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm" (OuterVolumeSpecName: "dshm") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:52:22.622752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.622725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:52:22.622752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.622738    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc" (OuterVolumeSpecName: "kube-api-access-m5snc") pod "f10203bd-a6cb-4be2-9a50-15e73a137351" (UID: "f10203bd-a6cb-4be2-9a50-15e73a137351"). InnerVolumeSpecName "kube-api-access-m5snc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:52:22.721118 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.721083    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5snc\" (UniqueName: \"kubernetes.io/projected/f10203bd-a6cb-4be2-9a50-15e73a137351-kube-api-access-m5snc\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:22.721118 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.721112    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f10203bd-a6cb-4be2-9a50-15e73a137351-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:22.721118 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:22.721123    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f10203bd-a6cb-4be2-9a50-15e73a137351-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:23.367660 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.367626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:23.391122 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.391092    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"
Jul 24 20:52:23.391263 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.391098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw" event={"ID":"f10203bd-a6cb-4be2-9a50-15e73a137351","Type":"ContainerDied","Data":"954934fc30c2675c8217d4656fa37790672d7c3304b01b22cf58698ee2413238"}
Jul 24 20:52:23.391263 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.391223    2577 scope.go:117] "RemoveContainer" containerID="97dff65955c0c06835873a0d4a739ea4e6adf30923898b018e5cfc35651f6497"
Jul 24 20:52:23.411043 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.411019    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:52:23.415164 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.415140    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c97e06/test-model-c97e06-kserve-9bc96d757-4g2tw"]
Jul 24 20:52:23.592114 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.592081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:52:23.797111 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:23.797072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:24.152567 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:24.152484    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f10203bd-a6cb-4be2-9a50-15e73a137351" path="/var/lib/kubelet/pods/f10203bd-a6cb-4be2-9a50-15e73a137351/volumes"
Jul 24 20:52:29.440556 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:29.440511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:29.540667 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:29.540621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:29.702684 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:29.702567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:29.938024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:29.937988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:30.078262 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078230    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"]
Jul 24 20:52:30.078643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078631    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f10203bd-a6cb-4be2-9a50-15e73a137351" containerName="main"
Jul 24 20:52:30.078704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078646    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f10203bd-a6cb-4be2-9a50-15e73a137351" containerName="main"
Jul 24 20:52:30.078704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078656    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" containerName="payload-pre-processing"
Jul 24 20:52:30.078704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078662    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" containerName="payload-pre-processing"
Jul 24 20:52:30.078704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078672    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" containerName="payload-processing"
Jul 24 20:52:30.078704 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078677    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" containerName="payload-processing"
Jul 24 20:52:30.078857 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078743    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f10203bd-a6cb-4be2-9a50-15e73a137351" containerName="main"
Jul 24 20:52:30.078857 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078754    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8306ef7-bb5a-4cf7-9333-3217e82c75e8" containerName="payload-processing"
Jul 24 20:52:30.078857 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.078761    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a31f94-2b1f-4ab9-b470-a7d9c793747d" containerName="payload-pre-processing"
Jul 24 20:52:30.083064 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.083043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:30.085490 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.085463    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 20:52:30.085633 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.085510    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-85rr2\""
Jul 24 20:52:30.088946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.088918    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"]
Jul 24 20:52:30.187793 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.187757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p\" (UID: \"62cb8982-0b88-4ea5-9436-6d9759be3355\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:30.288477 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.288448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p\" (UID: \"62cb8982-0b88-4ea5-9436-6d9759be3355\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:30.289089 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.289071    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p\" (UID: \"62cb8982-0b88-4ea5-9436-6d9759be3355\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:30.395938 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.395862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:30.527192 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.527165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"]
Jul 24 20:52:30.528200 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:52:30.528177    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cb8982_0b88_4ea5_9436_6d9759be3355.slice/crio-a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726 WatchSource:0}: Error finding container a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726: Status 404 returned error can't find the container with id a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726
Jul 24 20:52:30.530255 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:30.530234    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 20:52:31.435105 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:31.435064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p" event={"ID":"62cb8982-0b88-4ea5-9436-6d9759be3355","Type":"ContainerStarted","Data":"a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726"}
Jul 24 20:52:33.446257 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:33.446225    2577 generic.go:356] "Generic (PLEG): container finished" podID="62cb8982-0b88-4ea5-9436-6d9759be3355" containerID="59307bf0a13a69204dfe1dadf1e43756e545011f28a43453a931458260dd40ad" exitCode=0
Jul 24 20:52:33.446671 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:33.446299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p" event={"ID":"62cb8982-0b88-4ea5-9436-6d9759be3355","Type":"ContainerDied","Data":"59307bf0a13a69204dfe1dadf1e43756e545011f28a43453a931458260dd40ad"}
Jul 24 20:52:34.579798 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:34.579767    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:34.732820 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:34.732788    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca\") pod \"62cb8982-0b88-4ea5-9436-6d9759be3355\" (UID: \"62cb8982-0b88-4ea5-9436-6d9759be3355\") "
Jul 24 20:52:34.733187 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:34.733151    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62cb8982-0b88-4ea5-9436-6d9759be3355" (UID: "62cb8982-0b88-4ea5-9436-6d9759be3355"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:52:34.833864 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:34.833786    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62cb8982-0b88-4ea5-9436-6d9759be3355-maas-api-service-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:35.456373 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:35.456343    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"
Jul 24 20:52:35.456554 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:35.456342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p" event={"ID":"62cb8982-0b88-4ea5-9436-6d9759be3355","Type":"ContainerDied","Data":"a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726"}
Jul 24 20:52:35.456554 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:35.456469    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8c599dea4ccd24207864b7be2cd70d37e67234b072661a6f75fe85ad5f70726"
Jul 24 20:52:40.461560 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.461525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:40.493069 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.493037    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:52:40.493300 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.493274    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" podUID="f417a223-fd92-4b98-be01-adb805a34f88" containerName="payload-pre-processing" containerID="cri-o://73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2" gracePeriod=30
Jul 24 20:52:40.558325 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.558297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:40.615900 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.615873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:40.741339 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.741300    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:52:40.889223 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.889196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume\") pod \"f417a223-fd92-4b98-be01-adb805a34f88\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") "
Jul 24 20:52:40.889416 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.889281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vch8\" (UniqueName: \"kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8\") pod \"f417a223-fd92-4b98-be01-adb805a34f88\" (UID: \"f417a223-fd92-4b98-be01-adb805a34f88\") "
Jul 24 20:52:40.889733 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.889712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f417a223-fd92-4b98-be01-adb805a34f88" (UID: "f417a223-fd92-4b98-be01-adb805a34f88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:52:40.891613 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.891584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8" (OuterVolumeSpecName: "kube-api-access-2vch8") pod "f417a223-fd92-4b98-be01-adb805a34f88" (UID: "f417a223-fd92-4b98-be01-adb805a34f88"). InnerVolumeSpecName "kube-api-access-2vch8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:52:40.990552 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.990470    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f417a223-fd92-4b98-be01-adb805a34f88-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:40.990552 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:40.990507    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vch8\" (UniqueName: \"kubernetes.io/projected/f417a223-fd92-4b98-be01-adb805a34f88-kube-api-access-2vch8\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:52:41.488031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.487998    2577 generic.go:356] "Generic (PLEG): container finished" podID="f417a223-fd92-4b98-be01-adb805a34f88" containerID="73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2" exitCode=0
Jul 24 20:52:41.488446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.488043    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"
Jul 24 20:52:41.488446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.488086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" event={"ID":"f417a223-fd92-4b98-be01-adb805a34f88","Type":"ContainerDied","Data":"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"}
Jul 24 20:52:41.488446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.488140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5" event={"ID":"f417a223-fd92-4b98-be01-adb805a34f88","Type":"ContainerDied","Data":"9bf6f311cc06a1cd3840f279d9e837b1320c162b345c9d451403736dfd3a4d4e"}
Jul 24 20:52:41.488446 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.488166    2577 scope.go:117] "RemoveContainer" containerID="73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"
Jul 24 20:52:41.497673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.497475    2577 scope.go:117] "RemoveContainer" containerID="73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"
Jul 24 20:52:41.497777 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:52:41.497758    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2\": container with ID starting with 73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2 not found: ID does not exist" containerID="73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"
Jul 24 20:52:41.497815 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.497786    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2"} err="failed to get container status \"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2\": rpc error: code = NotFound desc = could not find container \"73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2\": container with ID starting with 73784628c45540c06b69fa2ac7cf70645e0391b60401f042501fef84c3d70fc2 not found: ID does not exist"
Jul 24 20:52:41.509286 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.509260    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:52:41.512980 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:41.512961    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-55d9b1-6d49566ff6-27kc5"]
Jul 24 20:52:42.153844 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:42.153813    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f417a223-fd92-4b98-be01-adb805a34f88" path="/var/lib/kubelet/pods/f417a223-fd92-4b98-be01-adb805a34f88/volumes"
Jul 24 20:52:43.294739 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:43.294701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:52:45.142830 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:45.142791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"]
Jul 24 20:52:45.145968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:45.145937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:45.146287 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:45.146269    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-55d9b1-7tp5p"]
Jul 24 20:52:45.542048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:45.542019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:46.151898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:46.151864    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62cb8982-0b88-4ea5-9436-6d9759be3355" path="/var/lib/kubelet/pods/62cb8982-0b88-4ea5-9436-6d9759be3355/volumes"
Jul 24 20:52:51.525062 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:51.525025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:51.626380 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:51.626346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:51.858510 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:51.858431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:52.082117 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:52.082085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:52.443472 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:52.443437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:52.758886 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:52.758857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:53.066491 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:53.066406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:55.912543 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:55.912501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:55.995958 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:55.995907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:56.058620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.058548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:56.188092 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.188016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:56.481853 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.481771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:56.574048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.574012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:52:56.580529 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.580496    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:52:56.581335 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581314    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62cb8982-0b88-4ea5-9436-6d9759be3355" containerName="revoke-keys"
Jul 24 20:52:56.581426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581338    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62cb8982-0b88-4ea5-9436-6d9759be3355" containerName="revoke-keys"
Jul 24 20:52:56.581426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581394    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f417a223-fd92-4b98-be01-adb805a34f88" containerName="payload-pre-processing"
Jul 24 20:52:56.581426 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581405    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f417a223-fd92-4b98-be01-adb805a34f88" containerName="payload-pre-processing"
Jul 24 20:52:56.581615 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581582    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f417a223-fd92-4b98-be01-adb805a34f88" containerName="payload-pre-processing"
Jul 24 20:52:56.581675 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.581631    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62cb8982-0b88-4ea5-9436-6d9759be3355" containerName="revoke-keys"
Jul 24 20:52:56.587276 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.587244    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.590773 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.590616    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-678134-dockercfg-dfvjt\""
Jul 24 20:52:56.590773 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.590643    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-678134\""
Jul 24 20:52:56.591794 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.591774    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:52:56.615752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.615722    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:52:56.618040 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.618015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdpnq\" (UniqueName: \"kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.618201 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.618143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.619374 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.619360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.628756 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.628726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:52:56.718481 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.718448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.718666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.718493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdpnq\" (UniqueName: \"kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.718666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.718517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.718666 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.718629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98f6v\" (UniqueName: \"kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.719154 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.719135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.727728 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.727707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdpnq\" (UniqueName: \"kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq\") pod \"payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.819643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.819589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.819791 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.819696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98f6v\" (UniqueName: \"kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.820182 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.820162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.827326 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.827293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98f6v\" (UniqueName: \"kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v\") pod \"payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") " pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:56.899726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.899692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:56.929233 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:56.929203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:57.106450 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.106422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:52:57.109192 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:52:57.109162    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48eae80_b199_43e8_a51e_a40d3118082d.slice/crio-9ffe332d6715ae7348ecc9b07eeea1812f64f81efc67e98db7d1956af29332d2 WatchSource:0}: Error finding container 9ffe332d6715ae7348ecc9b07eeea1812f64f81efc67e98db7d1956af29332d2: Status 404 returned error can't find the container with id 9ffe332d6715ae7348ecc9b07eeea1812f64f81efc67e98db7d1956af29332d2
Jul 24 20:52:57.261405 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.261379    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:52:57.265340 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:52:57.265302    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f0bb74_5b3e_4a03_b3ac_4d08c3c3694e.slice/crio-ee37b4813dad3b72f8421bf8d77cd9b8e3cbf6d0a3f7bcd36ebc0083728592f7 WatchSource:0}: Error finding container ee37b4813dad3b72f8421bf8d77cd9b8e3cbf6d0a3f7bcd36ebc0083728592f7: Status 404 returned error can't find the container with id ee37b4813dad3b72f8421bf8d77cd9b8e3cbf6d0a3f7bcd36ebc0083728592f7
Jul 24 20:52:57.567897 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.567858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" event={"ID":"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e","Type":"ContainerStarted","Data":"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"}
Jul 24 20:52:57.568133 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.567924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:52:57.568133 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.567950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" event={"ID":"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e","Type":"ContainerStarted","Data":"ee37b4813dad3b72f8421bf8d77cd9b8e3cbf6d0a3f7bcd36ebc0083728592f7"}
Jul 24 20:52:57.569333 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.569304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" event={"ID":"d48eae80-b199-43e8-a51e-a40d3118082d","Type":"ContainerStarted","Data":"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"}
Jul 24 20:52:57.569454 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.569344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" event={"ID":"d48eae80-b199-43e8-a51e-a40d3118082d","Type":"ContainerStarted","Data":"9ffe332d6715ae7348ecc9b07eeea1812f64f81efc67e98db7d1956af29332d2"}
Jul 24 20:52:57.569454 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.569364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:52:57.584211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.584170    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" podStartSLOduration=1.584156444 podStartE2EDuration="1.584156444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:52:57.582342195 +0000 UTC m=+2243.935919364" watchObservedRunningTime="2026-07-24 20:52:57.584156444 +0000 UTC m=+2243.937733583"
Jul 24 20:52:57.599415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:52:57.599371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" podStartSLOduration=1.599353864 podStartE2EDuration="1.599353864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:52:57.599260343 +0000 UTC m=+2243.952837530" watchObservedRunningTime="2026-07-24 20:52:57.599353864 +0000 UTC m=+2243.952931005"
Jul 24 20:53:00.399858 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:00.399829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:00.902457 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:00.902411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:01.248065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:01.247967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:01.741329 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:01.741297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:03.940741 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:03.940708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.021500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.021465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.096007 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.095975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.106646 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.106621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:53:04.195896 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.195814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.595535 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.595508    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.698355 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.698319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:04.705484 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.705455    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:53:04.709182 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.709164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.711998 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.711975    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1ae1a9-dockercfg-pcdt5\""
Jul 24 20:53:04.712110 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.712004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1ae1a9\""
Jul 24 20:53:04.718770 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.718749    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:53:04.741905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.741871    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:04.745344 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.745324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.753048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.753024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:04.790115 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.790084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.790231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.790125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwtc8\" (UniqueName: \"kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.790231 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.790211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjg5j\" (UniqueName: \"kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.790317 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.790248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.891506 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.891427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjg5j\" (UniqueName: \"kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.891506 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.891474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.891733 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.891514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.891733 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.891543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwtc8\" (UniqueName: \"kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.892338 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.892296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:04.892338 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.892317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.901163 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.901129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwtc8\" (UniqueName: \"kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8\") pod \"payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:04.901163 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:04.901161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjg5j\" (UniqueName: \"kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j\") pod \"payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:05.020574 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.020535    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:05.056111 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.056070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:05.167984 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.167950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:53:05.168887 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:53:05.168856    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1529af3_f47e_4c90_a6d1_8a9c21eda9ee.slice/crio-8ff956a9c837e017a4e56de3c31dd982e4fe65d89605c49ea05585e2e41efa2b WatchSource:0}: Error finding container 8ff956a9c837e017a4e56de3c31dd982e4fe65d89605c49ea05585e2e41efa2b: Status 404 returned error can't find the container with id 8ff956a9c837e017a4e56de3c31dd982e4fe65d89605c49ea05585e2e41efa2b
Jul 24 20:53:05.201968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.201905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:05.204143 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:53:05.204119    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a306166_a7db_44f4_bba3_c3b6aa4d84c7.slice/crio-03d6990252b12d951413f42d8d42e9a133365c1c80cd649248d258b17d97b208 WatchSource:0}: Error finding container 03d6990252b12d951413f42d8d42e9a133365c1c80cd649248d258b17d97b208: Status 404 returned error can't find the container with id 03d6990252b12d951413f42d8d42e9a133365c1c80cd649248d258b17d97b208
Jul 24 20:53:05.609211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.609176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" event={"ID":"4a306166-a7db-44f4-bba3-c3b6aa4d84c7","Type":"ContainerStarted","Data":"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"}
Jul 24 20:53:05.609407 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.609330    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:05.609407 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.609362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" event={"ID":"4a306166-a7db-44f4-bba3-c3b6aa4d84c7","Type":"ContainerStarted","Data":"03d6990252b12d951413f42d8d42e9a133365c1c80cd649248d258b17d97b208"}
Jul 24 20:53:05.610951 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.610910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" event={"ID":"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee","Type":"ContainerStarted","Data":"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"}
Jul 24 20:53:05.611079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.610968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:05.611079 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.610991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" event={"ID":"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee","Type":"ContainerStarted","Data":"8ff956a9c837e017a4e56de3c31dd982e4fe65d89605c49ea05585e2e41efa2b"}
Jul 24 20:53:05.626544 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.626492    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" podStartSLOduration=1.626479347 podStartE2EDuration="1.626479347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:53:05.624861926 +0000 UTC m=+2251.978439091" watchObservedRunningTime="2026-07-24 20:53:05.626479347 +0000 UTC m=+2251.980056537"
Jul 24 20:53:05.640196 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:05.640152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" podStartSLOduration=1.640134796 podStartE2EDuration="1.640134796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:53:05.638991947 +0000 UTC m=+2251.992569112" watchObservedRunningTime="2026-07-24 20:53:05.640134796 +0000 UTC m=+2251.993711939"
Jul 24 20:53:08.457970 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:08.457942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:08.576778 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:08.576745    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:53:08.577010 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:08.576994    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:53:16.619532 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:16.619497    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:16.620055 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:16.619577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:19.689673 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:19.689637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:53:32.435185 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.435149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:32.453171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.453136    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:53:32.453360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.453337    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" podUID="d48eae80-b199-43e8-a51e-a40d3118082d" containerName="payload-processing" containerID="cri-o://2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44" gracePeriod=30
Jul 24 20:53:32.460429 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.460386    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:53:32.460689 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.460631    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" podUID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" containerName="payload-pre-processing" containerID="cri-o://be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c" gracePeriod=30
Jul 24 20:53:32.534752 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.534716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:32.588358 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.588328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:32.705206 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.705183    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:53:32.732869 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.732846    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:53:32.743638 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.743611    2577 generic.go:356] "Generic (PLEG): container finished" podID="d48eae80-b199-43e8-a51e-a40d3118082d" containerID="2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44" exitCode=0
Jul 24 20:53:32.743736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.743664    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"
Jul 24 20:53:32.743736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.743680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" event={"ID":"d48eae80-b199-43e8-a51e-a40d3118082d","Type":"ContainerDied","Data":"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"}
Jul 24 20:53:32.743736 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.743725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f" event={"ID":"d48eae80-b199-43e8-a51e-a40d3118082d","Type":"ContainerDied","Data":"9ffe332d6715ae7348ecc9b07eeea1812f64f81efc67e98db7d1956af29332d2"}
Jul 24 20:53:32.743876 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.743754    2577 scope.go:117] "RemoveContainer" containerID="2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"
Jul 24 20:53:32.745071 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.745048    2577 generic.go:356] "Generic (PLEG): container finished" podID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" containerID="be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c" exitCode=0
Jul 24 20:53:32.745148 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.745085    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"
Jul 24 20:53:32.745211 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.745085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" event={"ID":"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e","Type":"ContainerDied","Data":"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"}
Jul 24 20:53:32.745252 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.745218    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6" event={"ID":"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e","Type":"ContainerDied","Data":"ee37b4813dad3b72f8421bf8d77cd9b8e3cbf6d0a3f7bcd36ebc0083728592f7"}
Jul 24 20:53:32.756055 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.755797    2577 scope.go:117] "RemoveContainer" containerID="2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"
Jul 24 20:53:32.757465 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:53:32.757424    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44\": container with ID starting with 2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44 not found: ID does not exist" containerID="2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"
Jul 24 20:53:32.757561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.757472    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44"} err="failed to get container status \"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44\": rpc error: code = NotFound desc = could not find container \"2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44\": container with ID starting with 2b86f439f95be8d0d311c3551b4fc0201840b63b9fb97b7f0e976fdf7a8f6c44 not found: ID does not exist"
Jul 24 20:53:32.757561 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.757495    2577 scope.go:117] "RemoveContainer" containerID="be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"
Jul 24 20:53:32.765653 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.765634    2577 scope.go:117] "RemoveContainer" containerID="be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"
Jul 24 20:53:32.765947 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:53:32.765925    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c\": container with ID starting with be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c not found: ID does not exist" containerID="be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"
Jul 24 20:53:32.765999 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.765954    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c"} err="failed to get container status \"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c\": rpc error: code = NotFound desc = could not find container \"be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c\": container with ID starting with be5408e0e575de29c906e593c3fd28e5b160beeab1d4e0fc0637dbc36b57ae7c not found: ID does not exist"
Jul 24 20:53:32.853924 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.853891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume\") pod \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") "
Jul 24 20:53:32.853924 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.853928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume\") pod \"d48eae80-b199-43e8-a51e-a40d3118082d\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") "
Jul 24 20:53:32.854139 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.853958    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98f6v\" (UniqueName: \"kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v\") pod \"d48eae80-b199-43e8-a51e-a40d3118082d\" (UID: \"d48eae80-b199-43e8-a51e-a40d3118082d\") "
Jul 24 20:53:32.854139 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.853986    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdpnq\" (UniqueName: \"kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq\") pod \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\" (UID: \"76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e\") "
Jul 24 20:53:32.854282 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.854259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d48eae80-b199-43e8-a51e-a40d3118082d" (UID: "d48eae80-b199-43e8-a51e-a40d3118082d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:53:32.854334 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.854286    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" (UID: "76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:53:32.856112 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.856079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v" (OuterVolumeSpecName: "kube-api-access-98f6v") pod "d48eae80-b199-43e8-a51e-a40d3118082d" (UID: "d48eae80-b199-43e8-a51e-a40d3118082d"). InnerVolumeSpecName "kube-api-access-98f6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:53:32.856199 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.856154    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq" (OuterVolumeSpecName: "kube-api-access-rdpnq") pod "76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" (UID: "76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e"). InnerVolumeSpecName "kube-api-access-rdpnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:53:32.954928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.954861    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:32.954928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.954884    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d48eae80-b199-43e8-a51e-a40d3118082d-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:32.954928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.954893    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98f6v\" (UniqueName: \"kubernetes.io/projected/d48eae80-b199-43e8-a51e-a40d3118082d-kube-api-access-98f6v\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:32.954928 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:32.954901    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdpnq\" (UniqueName: \"kubernetes.io/projected/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e-kube-api-access-rdpnq\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:33.068542 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:33.068513    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:53:33.072532 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:33.072498    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-678134-678bf54484-bs9n6"]
Jul 24 20:53:33.082547 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:33.082523    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:53:33.088202 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:33.088181    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-678134-744dc4bfd-xkm7f"]
Jul 24 20:53:34.152353 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:34.152324    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" path="/var/lib/kubelet/pods/76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e/volumes"
Jul 24 20:53:34.152722 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:34.152698    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d48eae80-b199-43e8-a51e-a40d3118082d" path="/var/lib/kubelet/pods/d48eae80-b199-43e8-a51e-a40d3118082d/volumes"
Jul 24 20:53:35.590629 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:35.589520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:53:36.739714 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:36.739676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:37.140005 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:37.139966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:42.817750 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:42.817715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:42.878023 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:42.877972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:43.093558 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.093463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:43.347080 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.346988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:43.475438 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475401    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"]
Jul 24 20:53:43.475810 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475798    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d48eae80-b199-43e8-a51e-a40d3118082d" containerName="payload-processing"
Jul 24 20:53:43.475863 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475812    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d48eae80-b199-43e8-a51e-a40d3118082d" containerName="payload-processing"
Jul 24 20:53:43.475863 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475827    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" containerName="payload-pre-processing"
Jul 24 20:53:43.475863 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475832    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" containerName="payload-pre-processing"
Jul 24 20:53:43.475997 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475885    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d48eae80-b199-43e8-a51e-a40d3118082d" containerName="payload-processing"
Jul 24 20:53:43.475997 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.475896    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f0bb74-5b3e-4a03-b3ac-4d08c3c3694e" containerName="payload-pre-processing"
Jul 24 20:53:43.479093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.479075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:43.481744 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.481717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-85rr2\""
Jul 24 20:53:43.481847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.481748    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 20:53:43.485779 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.485753    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"]
Jul 24 20:53:43.548409 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.548375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w\" (UID: \"966c09aa-39d7-4b14-b97e-8387ee904286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:43.649548 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.649468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w\" (UID: \"966c09aa-39d7-4b14-b97e-8387ee904286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:43.650138 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.650118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w\" (UID: \"966c09aa-39d7-4b14-b97e-8387ee904286\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:43.790238 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.790203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:43.924945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:43.924909    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"]
Jul 24 20:53:43.927463 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:53:43.927436    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966c09aa_39d7_4b14_b97e_8387ee904286.slice/crio-3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9 WatchSource:0}: Error finding container 3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9: Status 404 returned error can't find the container with id 3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9
Jul 24 20:53:44.806476 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:44.806388    2577 generic.go:356] "Generic (PLEG): container finished" podID="966c09aa-39d7-4b14-b97e-8387ee904286" containerID="511ed5f4faae8fb4367944862fa9adce095c09a2b43e1a16c213c0ac95340ca6" exitCode=0
Jul 24 20:53:44.806644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:44.806480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w" event={"ID":"966c09aa-39d7-4b14-b97e-8387ee904286","Type":"ContainerDied","Data":"511ed5f4faae8fb4367944862fa9adce095c09a2b43e1a16c213c0ac95340ca6"}
Jul 24 20:53:44.806644 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:44.806531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w" event={"ID":"966c09aa-39d7-4b14-b97e-8387ee904286","Type":"ContainerStarted","Data":"3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9"}
Jul 24 20:53:45.938525 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:45.938497    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:45.970331 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:45.970307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca\") pod \"966c09aa-39d7-4b14-b97e-8387ee904286\" (UID: \"966c09aa-39d7-4b14-b97e-8387ee904286\") "
Jul 24 20:53:45.970703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:45.970681    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "966c09aa-39d7-4b14-b97e-8387ee904286" (UID: "966c09aa-39d7-4b14-b97e-8387ee904286"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:53:46.071855 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:46.071774    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/966c09aa-39d7-4b14-b97e-8387ee904286-maas-api-service-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:46.816911 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:46.816878    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"
Jul 24 20:53:46.817090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:46.816912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w" event={"ID":"966c09aa-39d7-4b14-b97e-8387ee904286","Type":"ContainerDied","Data":"3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9"}
Jul 24 20:53:46.817090 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:46.816957    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3feaa7de9ce9bf06e5fac114b74bd119aac7d931cca9a7c5cce022504124c9b9"
Jul 24 20:53:54.834753 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:54.834716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:55.984973 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:55.984939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:58.861847 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.861810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:58.887949 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.887917    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:58.888177 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.888152    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" podUID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" containerName="payload-processing" containerID="cri-o://5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80" gracePeriod=30
Jul 24 20:53:58.894950 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.894920    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:53:58.895174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.895135    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" podUID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" containerName="payload-pre-processing" containerID="cri-o://0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818" gracePeriod=30
Jul 24 20:53:58.929918 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.929889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:58.986850 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:58.986819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:53:59.150380 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.150352    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:59.174013 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.173990    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:59.191612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.191538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjg5j\" (UniqueName: \"kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j\") pod \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") "
Jul 24 20:53:59.191612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.191584    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume\") pod \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\" (UID: \"4a306166-a7db-44f4-bba3-c3b6aa4d84c7\") "
Jul 24 20:53:59.192017 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.191992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a306166-a7db-44f4-bba3-c3b6aa4d84c7" (UID: "4a306166-a7db-44f4-bba3-c3b6aa4d84c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:53:59.193810 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.193793    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j" (OuterVolumeSpecName: "kube-api-access-xjg5j") pod "4a306166-a7db-44f4-bba3-c3b6aa4d84c7" (UID: "4a306166-a7db-44f4-bba3-c3b6aa4d84c7"). InnerVolumeSpecName "kube-api-access-xjg5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:53:59.292401 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.292372    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume\") pod \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") "
Jul 24 20:53:59.292537 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.292418    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwtc8\" (UniqueName: \"kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8\") pod \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\" (UID: \"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee\") "
Jul 24 20:53:59.292734 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.292715    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjg5j\" (UniqueName: \"kubernetes.io/projected/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-kube-api-access-xjg5j\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:59.292779 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.292755    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a306166-a7db-44f4-bba3-c3b6aa4d84c7-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:59.292826 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.292810    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" (UID: "c1529af3-f47e-4c90-a6d1-8a9c21eda9ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 20:53:59.294557 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.294537    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8" (OuterVolumeSpecName: "kube-api-access-nwtc8") pod "c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" (UID: "c1529af3-f47e-4c90-a6d1-8a9c21eda9ee"). InnerVolumeSpecName "kube-api-access-nwtc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:53:59.393762 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.393694    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:59.393762 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.393716    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwtc8\" (UniqueName: \"kubernetes.io/projected/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee-kube-api-access-nwtc8\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:53:59.886155 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.886115    2577 generic.go:356] "Generic (PLEG): container finished" podID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" containerID="5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80" exitCode=0
Jul 24 20:53:59.886612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.886177    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"
Jul 24 20:53:59.886612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.886196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" event={"ID":"4a306166-a7db-44f4-bba3-c3b6aa4d84c7","Type":"ContainerDied","Data":"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"}
Jul 24 20:53:59.886612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.886284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff" event={"ID":"4a306166-a7db-44f4-bba3-c3b6aa4d84c7","Type":"ContainerDied","Data":"03d6990252b12d951413f42d8d42e9a133365c1c80cd649248d258b17d97b208"}
Jul 24 20:53:59.886612 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.886312    2577 scope.go:117] "RemoveContainer" containerID="5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"
Jul 24 20:53:59.887468 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.887448    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" containerID="0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818" exitCode=0
Jul 24 20:53:59.887575 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.887491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" event={"ID":"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee","Type":"ContainerDied","Data":"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"}
Jul 24 20:53:59.887575 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.887505    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"
Jul 24 20:53:59.887575 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.887526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7" event={"ID":"c1529af3-f47e-4c90-a6d1-8a9c21eda9ee","Type":"ContainerDied","Data":"8ff956a9c837e017a4e56de3c31dd982e4fe65d89605c49ea05585e2e41efa2b"}
Jul 24 20:53:59.895618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.895389    2577 scope.go:117] "RemoveContainer" containerID="5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"
Jul 24 20:53:59.895775 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:53:59.895756    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80\": container with ID starting with 5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80 not found: ID does not exist" containerID="5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"
Jul 24 20:53:59.895848 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.895784    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80"} err="failed to get container status \"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80\": rpc error: code = NotFound desc = could not find container \"5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80\": container with ID starting with 5c1cc5a1324bf6ae603b69674cb0eeb448cf0c41233ec56f7fb46d0dd5157f80 not found: ID does not exist"
Jul 24 20:53:59.895848 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.895800    2577 scope.go:117] "RemoveContainer" containerID="0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"
Jul 24 20:53:59.911047 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.911022    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:59.911687 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.911669    2577 scope.go:117] "RemoveContainer" containerID="0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"
Jul 24 20:53:59.911936 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:53:59.911918    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818\": container with ID starting with 0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818 not found: ID does not exist" containerID="0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"
Jul 24 20:53:59.911987 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.911944    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818"} err="failed to get container status \"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818\": rpc error: code = NotFound desc = could not find container \"0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818\": container with ID starting with 0db43eb2cdd38a73f28cb860d6e228edec41db6351545def44ac5ea57f81f818 not found: ID does not exist"
Jul 24 20:53:59.915878 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.915858    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ae1a9-7cf7894cb6-jnvff"]
Jul 24 20:53:59.926171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.926144    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:53:59.931726 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:53:59.931707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ae1a9-b46754584-7khj7"]
Jul 24 20:54:00.153271 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:00.153194    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" path="/var/lib/kubelet/pods/4a306166-a7db-44f4-bba3-c3b6aa4d84c7/volumes"
Jul 24 20:54:00.153569 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:00.153556    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" path="/var/lib/kubelet/pods/c1529af3-f47e-4c90-a6d1-8a9c21eda9ee/volumes"
Jul 24 20:54:03.289408 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:03.289371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:54:03.543895 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:03.543814    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"]
Jul 24 20:54:03.546532 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:03.546506    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ae1a9-r5s6w"]
Jul 24 20:54:03.549057 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:03.549033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:03.925020 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:03.924934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:04.151906 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:04.151873    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966c09aa-39d7-4b14-b97e-8387ee904286" path="/var/lib/kubelet/pods/966c09aa-39d7-4b14-b97e-8387ee904286/volumes"
Jul 24 20:54:09.474516 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:09.474478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:09.572125 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:09.572056    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:09.754983 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:09.754901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:09.978245 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:09.978208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:10.345084 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:10.345051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:10.654952 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:10.654865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:11.100613 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:11.100555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.284135 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.284085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.356388 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.356359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.434018 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.433984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.705789 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.705678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.836093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.836059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.934313 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.934250    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:13.938680 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.938653    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 20:54:13.939150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939133    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" containerName="payload-pre-processing"
Jul 24 20:54:13.939150 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939151    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" containerName="payload-pre-processing"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939169    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" containerName="payload-processing"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939175    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" containerName="payload-processing"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939180    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966c09aa-39d7-4b14-b97e-8387ee904286" containerName="revoke-keys"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939186    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="966c09aa-39d7-4b14-b97e-8387ee904286" containerName="revoke-keys"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939249    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1529af3-f47e-4c90-a6d1-8a9c21eda9ee" containerName="payload-pre-processing"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939259    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="966c09aa-39d7-4b14-b97e-8387ee904286" containerName="revoke-keys"
Jul 24 20:54:13.939268 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.939266    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a306166-a7db-44f4-bba3-c3b6aa4d84c7" containerName="payload-processing"
Jul 24 20:54:13.942643 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.942624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:13.945447 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.945428    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-3f62e6-dockercfg-8jsjb\""
Jul 24 20:54:13.945447 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.945440    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-3f62e6\""
Jul 24 20:54:13.952878 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.952859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 20:54:13.980526 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.980462    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 20:54:13.983819 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.983804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:13.993414 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:13.993390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 20:54:14.023324 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.023297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.023475 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.023342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpgg4\" (UniqueName: \"kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.124771 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.124740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.124926 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.124777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjwzf\" (UniqueName: \"kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.124926 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.124845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.124926 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.124874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpgg4\" (UniqueName: \"kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.125487 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.125468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.136811 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.136788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpgg4\" (UniqueName: \"kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4\") pod \"payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.226081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.226048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.226081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.226088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjwzf\" (UniqueName: \"kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.226852 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.226829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.234061 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.234010    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjwzf\" (UniqueName: \"kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf\") pod \"payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.253515 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.253493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.294944 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.294914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.396955 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.396706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 20:54:14.399236 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:54:14.399204    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745dd733_36cf_4a35_ae74_18999d369056.slice/crio-b45e71faa8e9163ea3175158b82e9874ff988b338c46dae12e89cd528b3774ca WatchSource:0}: Error finding container b45e71faa8e9163ea3175158b82e9874ff988b338c46dae12e89cd528b3774ca: Status 404 returned error can't find the container with id b45e71faa8e9163ea3175158b82e9874ff988b338c46dae12e89cd528b3774ca
Jul 24 20:54:14.440891 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.440864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 20:54:14.443735 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:54:14.443709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a000ee_759e_443e_83ac_6ba75c6f0051.slice/crio-867e747fffc8bac21deaf37a1b9020e692b4a4150736070b5ed38f1467f0ff44 WatchSource:0}: Error finding container 867e747fffc8bac21deaf37a1b9020e692b4a4150736070b5ed38f1467f0ff44: Status 404 returned error can't find the container with id 867e747fffc8bac21deaf37a1b9020e692b4a4150736070b5ed38f1467f0ff44
Jul 24 20:54:14.963317 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.963282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" event={"ID":"745dd733-36cf-4a35-ae74-18999d369056","Type":"ContainerStarted","Data":"6cd716fe4d0aa0cd4f8ecc8547823eb0c30e149aebcb7e75435a35225541a57e"}
Jul 24 20:54:14.963524 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.963348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" event={"ID":"745dd733-36cf-4a35-ae74-18999d369056","Type":"ContainerStarted","Data":"b45e71faa8e9163ea3175158b82e9874ff988b338c46dae12e89cd528b3774ca"}
Jul 24 20:54:14.963524 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.963372    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:14.964905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.964874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" event={"ID":"c0a000ee-759e-443e-83ac-6ba75c6f0051","Type":"ContainerStarted","Data":"95a6cc3b2595e3c21f386e565653e0df11ee1c762816f4423a65db3cc9c501be"}
Jul 24 20:54:14.965014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.964921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" event={"ID":"c0a000ee-759e-443e-83ac-6ba75c6f0051","Type":"ContainerStarted","Data":"867e747fffc8bac21deaf37a1b9020e692b4a4150736070b5ed38f1467f0ff44"}
Jul 24 20:54:14.965014 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.964944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:14.980422 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.980376    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" podStartSLOduration=1.980361413 podStartE2EDuration="1.980361413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:54:14.978629423 +0000 UTC m=+2321.332206591" watchObservedRunningTime="2026-07-24 20:54:14.980361413 +0000 UTC m=+2321.333938556"
Jul 24 20:54:14.994123 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:14.994079    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" podStartSLOduration=1.994065606 podStartE2EDuration="1.994065606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:54:14.992905556 +0000 UTC m=+2321.346482697" watchObservedRunningTime="2026-07-24 20:54:14.994065606 +0000 UTC m=+2321.347642785"
Jul 24 20:54:17.823368 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:17.823331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:18.181467 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:18.181385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:18.497906 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:18.497827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:18.680415 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:18.680382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:20.292448 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:20.292415    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:54:21.434041 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:21.434011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:21.503712 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:21.503681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:21.587024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:21.586991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:21.906659 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:21.906626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:22.231549 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.231471    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:22.328784 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.328757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:22.336968 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.336936    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 20:54:22.340685 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.340670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.343205 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.343182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-449ecf\""
Jul 24 20:54:22.343205 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.343199    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-449ecf-dockercfg-77g7d\""
Jul 24 20:54:22.349905 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.349877    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 20:54:22.370325 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.370300    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 20:54:22.373769 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.373754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.388999 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.388977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 20:54:22.503469 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.503392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.503469 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.503452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqskd\" (UniqueName: \"kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.503940 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.503487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.503940 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.503504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4886\" (UniqueName: \"kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.604790 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.604758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.604953 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.604840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqskd\" (UniqueName: \"kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.604953 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.604864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.604953 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.604881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4886\" (UniqueName: \"kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.605485 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.605460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.605635 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.605578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.613082 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.613060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqskd\" (UniqueName: \"kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd\") pod \"payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.613819 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.613799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4886\" (UniqueName: \"kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886\") pod \"payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") " pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.652111 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.652081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:22.683946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.683916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:22.795642 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:22.795612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 20:54:22.798973 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:54:22.798942    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e385f26_976a_4c2e_a21e_6cee70ce1df0.slice/crio-8e75a139b13c97ca9716231540edb27bba47444ce2048eb8f55f06260b442f77 WatchSource:0}: Error finding container 8e75a139b13c97ca9716231540edb27bba47444ce2048eb8f55f06260b442f77: Status 404 returned error can't find the container with id 8e75a139b13c97ca9716231540edb27bba47444ce2048eb8f55f06260b442f77
Jul 24 20:54:23.004477 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:23.004436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" event={"ID":"6e385f26-976a-4c2e-a21e-6cee70ce1df0","Type":"ContainerStarted","Data":"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"}
Jul 24 20:54:23.004683 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:23.004500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" event={"ID":"6e385f26-976a-4c2e-a21e-6cee70ce1df0","Type":"ContainerStarted","Data":"8e75a139b13c97ca9716231540edb27bba47444ce2048eb8f55f06260b442f77"}
Jul 24 20:54:23.004683 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:23.004533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:23.044052 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:23.038862    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" podStartSLOduration=1.038837361 podStartE2EDuration="1.038837361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:54:23.036411109 +0000 UTC m=+2329.389988248" watchObservedRunningTime="2026-07-24 20:54:23.038837361 +0000 UTC m=+2329.392414502"
Jul 24 20:54:23.050033 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:23.049944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 20:54:23.052285 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:54:23.052254    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8752ab26_8e28_4a29_8e28_0b180c0351e2.slice/crio-c1c373c2a0c6fdeba7523c3ef959487af46eda95c26b091e5865da6dc30f0e72 WatchSource:0}: Error finding container c1c373c2a0c6fdeba7523c3ef959487af46eda95c26b091e5865da6dc30f0e72: Status 404 returned error can't find the container with id c1c373c2a0c6fdeba7523c3ef959487af46eda95c26b091e5865da6dc30f0e72
Jul 24 20:54:24.012383 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:24.012336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" event={"ID":"8752ab26-8e28-4a29-8e28-0b180c0351e2","Type":"ContainerStarted","Data":"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"}
Jul 24 20:54:24.012843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:24.012406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" event={"ID":"8752ab26-8e28-4a29-8e28-0b180c0351e2","Type":"ContainerStarted","Data":"c1c373c2a0c6fdeba7523c3ef959487af46eda95c26b091e5865da6dc30f0e72"}
Jul 24 20:54:24.012843 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:24.012669    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:24.028749 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:24.028701    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" podStartSLOduration=2.028685025 podStartE2EDuration="2.028685025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:54:24.027581546 +0000 UTC m=+2330.381158686" watchObservedRunningTime="2026-07-24 20:54:24.028685025 +0000 UTC m=+2330.382262206"
Jul 24 20:54:25.969620 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:25.969527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:25.972129 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:25.972107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 20:54:25.972271 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:25.972170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 20:54:34.014081 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:34.014052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 20:54:34.192691 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:34.192660    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:54:35.019249 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:35.019219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 20:54:35.422674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:35.422579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:43.785237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:43.785199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:54:57.410833 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.410797    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:54:57.415839 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.415818    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.419709 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.419688    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"openshift-service-ca.crt\""
Jul 24 20:54:57.419828 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.419729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"default-dockercfg-zb9rm\""
Jul 24 20:54:57.419893 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.419821    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"kube-root-ca.crt\""
Jul 24 20:54:57.419893 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.419845    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"auth-test-model-449ecf-kserve-self-signed-certs\""
Jul 24 20:54:57.422681 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.422658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:54:57.437795 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.437763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:54:57.505815 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.505786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.505945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.505840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.505945 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.505915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.506028 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.505969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.506028 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.505999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.506101 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.506037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqmcs\" (UniqueName: \"kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606474 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.606720 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.606664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqmcs\" (UniqueName: \"kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.607046 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.607021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.607118 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.607060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.607118 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.607084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.609002 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.608980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.609292 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.609274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.615254 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.615232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqmcs\" (UniqueName: \"kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs\") pod \"auth-test-model-449ecf-kserve-5f56bdcdc-szfff\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") " pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.726031 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.725961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:54:57.858794 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:57.858766    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:54:57.860666 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:54:57.860637    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66173503_4bb6_4aaa_a480_8d5f34e5cf01.slice/crio-acdaab84e946799dd0183832432a40cd700053db34de982222d93fa9eb1a5800 WatchSource:0}: Error finding container acdaab84e946799dd0183832432a40cd700053db34de982222d93fa9eb1a5800: Status 404 returned error can't find the container with id acdaab84e946799dd0183832432a40cd700053db34de982222d93fa9eb1a5800
Jul 24 20:54:58.177768 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:58.177736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" event={"ID":"66173503-4bb6-4aaa-a480-8d5f34e5cf01","Type":"ContainerStarted","Data":"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"}
Jul 24 20:54:58.177768 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:58.177774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" event={"ID":"66173503-4bb6-4aaa-a480-8d5f34e5cf01","Type":"ContainerStarted","Data":"acdaab84e946799dd0183832432a40cd700053db34de982222d93fa9eb1a5800"}
Jul 24 20:54:58.198171 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:54:58.198116    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" podStartSLOduration=1.1980997150000001 podStartE2EDuration="1.198099715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:54:58.196652247 +0000 UTC m=+2364.550229387" watchObservedRunningTime="2026-07-24 20:54:58.198099715 +0000 UTC m=+2364.551676855"
Jul 24 20:55:00.891494 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:00.891458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:55:07.726986 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:07.726945    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:55:07.726986 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:07.726992    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:55:07.739265 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:07.739242    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:55:08.238914 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:08.238885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:55:22.768725 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:22.768691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:55:22.869491 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:22.869457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:55:27.582774 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:27.582738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:55:48.094703 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:55:48.094671    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:56:01.753237 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:56:01.753196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:56:02.494734 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:56:02.494699    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:56:24.091870 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:56:24.091835    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:56:40.288287 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:56:40.288250    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:56:42.240641 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:56:42.240541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:57:00.692362 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:57:00.692319    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:57:15.896898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:57:15.896861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:57:36.888194 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:57:36.888113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:57:54.192065 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:57:54.192028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:58:11.766402 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:11.766346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:58:17.700323 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:17.700290    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:58:21.393924 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:21.393876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:58:34.375220 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:34.375189    2577 scope.go:117] "RemoveContainer" containerID="59307bf0a13a69204dfe1dadf1e43756e545011f28a43453a931458260dd40ad"
Jul 24 20:58:34.388429 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:34.388405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:58:55.386086 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:58:55.386011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:59:00.029491 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:00.029459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:59:00.893839 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:00.893807    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:59:01.639564 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:01.639530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:59:11.097500 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:11.097468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:59:11.891889 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:11.891853    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:59:15.913523 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:15.913460    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" podUID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" containerName="main" containerID="cri-o://4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528" gracePeriod=45
Jul 24 20:59:16.168670 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.168583    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:59:16.232618 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232559    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.232795 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.232795 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.232795 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.232795 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232772    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache" (OuterVolumeSpecName: "model-cache") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:59:16.233032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232803    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqmcs\" (UniqueName: \"kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.233032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs\") pod \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\" (UID: \"66173503-4bb6-4aaa-a480-8d5f34e5cf01\") "
Jul 24 20:59:16.233032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232910    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home" (OuterVolumeSpecName: "home") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:59:16.233032 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.232997    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:59:16.233218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.233155    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.233218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.233170    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.233218 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.233179    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.235093 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.235072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 20:59:16.235173 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.235072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm" (OuterVolumeSpecName: "dshm") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 20:59:16.235173 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.235126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs" (OuterVolumeSpecName: "kube-api-access-dqmcs") pod "66173503-4bb6-4aaa-a480-8d5f34e5cf01" (UID: "66173503-4bb6-4aaa-a480-8d5f34e5cf01"). InnerVolumeSpecName "kube-api-access-dqmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 20:59:16.334247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.334217    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqmcs\" (UniqueName: \"kubernetes.io/projected/66173503-4bb6-4aaa-a480-8d5f34e5cf01-kube-api-access-dqmcs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.334247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.334246    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66173503-4bb6-4aaa-a480-8d5f34e5cf01-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.334247 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.334256    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66173503-4bb6-4aaa-a480-8d5f34e5cf01-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 20:59:16.425455 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.425362    2577 generic.go:356] "Generic (PLEG): container finished" podID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" containerID="4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528" exitCode=0
Jul 24 20:59:16.425455 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.425411    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"
Jul 24 20:59:16.425800 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.425450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" event={"ID":"66173503-4bb6-4aaa-a480-8d5f34e5cf01","Type":"ContainerDied","Data":"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"}
Jul 24 20:59:16.425800 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.425492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff" event={"ID":"66173503-4bb6-4aaa-a480-8d5f34e5cf01","Type":"ContainerDied","Data":"acdaab84e946799dd0183832432a40cd700053db34de982222d93fa9eb1a5800"}
Jul 24 20:59:16.425800 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.425509    2577 scope.go:117] "RemoveContainer" containerID="4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"
Jul 24 20:59:16.434971 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.434951    2577 scope.go:117] "RemoveContainer" containerID="4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"
Jul 24 20:59:16.437895 ip-10-0-136-110 kubenswrapper[2577]: E0724 20:59:16.437865    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528\": container with ID starting with 4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528 not found: ID does not exist" containerID="4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"
Jul 24 20:59:16.438016 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.437903    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528"} err="failed to get container status \"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528\": rpc error: code = NotFound desc = could not find container \"4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528\": container with ID starting with 4bf7b051de4721c6db3d57fe4f58da3e893e5c7e97e56d71ff54f8f1cab6d528 not found: ID does not exist"
Jul 24 20:59:16.448429 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.448407    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:59:16.451812 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:16.451790    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/auth-test-model-449ecf-kserve-5f56bdcdc-szfff"]
Jul 24 20:59:18.152432 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:18.152387    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" path="/var/lib/kubelet/pods/66173503-4bb6-4aaa-a480-8d5f34e5cf01/volumes"
Jul 24 20:59:22.287312 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:22.287281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:59:33.899024 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.898982    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 20:59:33.899455 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.899379    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" containerName="main"
Jul 24 20:59:33.899455 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.899390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" containerName="main"
Jul 24 20:59:33.899551 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.899485    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="66173503-4bb6-4aaa-a480-8d5f34e5cf01" containerName="main"
Jul 24 20:59:33.903946 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.903925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.907674 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.907653    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"openshift-service-ca.crt\""
Jul 24 20:59:33.907805 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.907680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"sub-test-model-449ecf-kserve-self-signed-certs\""
Jul 24 20:59:33.907805 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.907706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"kube-root-ca.crt\""
Jul 24 20:59:33.907805 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.907681    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"default-dockercfg-zb9rm\""
Jul 24 20:59:33.912306 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.912288    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 20:59:33.924029 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.924003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 20:59:33.987037 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgkdw\" (UniqueName: \"kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.987158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.987158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.987158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.987158 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:33.987297 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:33.987197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087724 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgkdw\" (UniqueName: \"kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.087898 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.087863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.088161 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.088113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.088219 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.088153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.088318 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.088299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.090226 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.090208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.090338 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.090323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"sub-test-model-449ecf-kserve-self-signed-certs\""
Jul 24 20:59:34.096174 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.096155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"kube-root-ca.crt\""
Jul 24 20:59:34.100408 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.100389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.106309 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.106288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"openshift-service-ca.crt\""
Jul 24 20:59:34.116491 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.116470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgkdw\" (UniqueName: \"kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw\") pod \"sub-test-model-449ecf-kserve-bb65c45c7-nxtnt\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") " pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.216163 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.216093    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-449ecf\"/\"default-dockercfg-zb9rm\""
Jul 24 20:59:34.224527 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.224506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:34.361220 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.361193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 20:59:34.362992 ip-10-0-136-110 kubenswrapper[2577]: W0724 20:59:34.362962    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44733221_8db1_484a_9df9_ebc53158426c.slice/crio-21e34b08bb297b1ca931bb771cf739a93c0519ae8bef9f02832eb29ec08579ce WatchSource:0}: Error finding container 21e34b08bb297b1ca931bb771cf739a93c0519ae8bef9f02832eb29ec08579ce: Status 404 returned error can't find the container with id 21e34b08bb297b1ca931bb771cf739a93c0519ae8bef9f02832eb29ec08579ce
Jul 24 20:59:34.516503 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.516470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" event={"ID":"44733221-8db1-484a-9df9-ebc53158426c","Type":"ContainerStarted","Data":"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"}
Jul 24 20:59:34.516682 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.516511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" event={"ID":"44733221-8db1-484a-9df9-ebc53158426c","Type":"ContainerStarted","Data":"21e34b08bb297b1ca931bb771cf739a93c0519ae8bef9f02832eb29ec08579ce"}
Jul 24 20:59:34.533869 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:34.533824    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" podStartSLOduration=1.533809725 podStartE2EDuration="1.533809725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 20:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 20:59:34.531258021 +0000 UTC m=+2640.884835172" watchObservedRunningTime="2026-07-24 20:59:34.533809725 +0000 UTC m=+2640.887386896"
Jul 24 20:59:37.791048 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:37.791005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 20:59:44.225275 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:44.225240    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:44.225275 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:44.225285    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:44.238723 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:44.238697    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 20:59:44.577360 ip-10-0-136-110 kubenswrapper[2577]: I0724 20:59:44.577331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 21:00:00.132115 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.132074    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"]
Jul 24 21:00:00.135749 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.135727    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"]
Jul 24 21:00:00.135895 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.135843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"
Jul 24 21:00:00.138137 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.138105    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-85rr2\""
Jul 24 21:00:00.139160 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.139139    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"
Jul 24 21:00:00.145459 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.145340    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"]
Jul 24 21:00:00.151455 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.151421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"
Jul 24 21:00:00.164226 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.164202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"]
Jul 24 21:00:00.165125 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.165098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"
Jul 24 21:00:00.307953 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.307925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"]
Jul 24 21:00:00.310189 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:00:00.310089    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c855c9_0331_4a84_9d90_72c17fd990e9.slice/crio-f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c WatchSource:0}: Error finding container f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c: Status 404 returned error can't find the container with id f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c
Jul 24 21:00:00.540268 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.540240    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"]
Jul 24 21:00:00.541840 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:00:00.541808    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc499f7_dbd1_47ac_a99c_59f38523007a.slice/crio-d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299 WatchSource:0}: Error finding container d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299: Status 404 returned error can't find the container with id d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299
Jul 24 21:00:00.643654 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.643625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst" event={"ID":"09c855c9-0331-4a84-9d90-72c17fd990e9","Type":"ContainerStarted","Data":"9653f456152f92425a47c95f785070b0306c9fa97d12fbf6a44a908eaa2650bc"}
Jul 24 21:00:00.643787 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.643682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst" event={"ID":"09c855c9-0331-4a84-9d90-72c17fd990e9","Type":"ContainerStarted","Data":"f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c"}
Jul 24 21:00:00.644789 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.644762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5" event={"ID":"5fc499f7-dbd1-47ac-a99c-59f38523007a","Type":"ContainerStarted","Data":"d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299"}
Jul 24 21:00:00.659221 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:00.659176    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst" podStartSLOduration=0.65915995 podStartE2EDuration="659.15995ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:00:00.657228443 +0000 UTC m=+2667.010805583" watchObservedRunningTime="2026-07-24 21:00:00.65915995 +0000 UTC m=+2667.012737089"
Jul 24 21:00:01.651237 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:01.651208    2577 generic.go:356] "Generic (PLEG): container finished" podID="5fc499f7-dbd1-47ac-a99c-59f38523007a" containerID="60b202ffced74c2a8a682870dbbb0d760bbadb8694571b1bbbab71f898094fb5" exitCode=0
Jul 24 21:00:01.651696 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:01.651251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5" event={"ID":"5fc499f7-dbd1-47ac-a99c-59f38523007a","Type":"ContainerDied","Data":"60b202ffced74c2a8a682870dbbb0d760bbadb8694571b1bbbab71f898094fb5"}
Jul 24 21:00:01.652789 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:01.652767    2577 generic.go:356] "Generic (PLEG): container finished" podID="09c855c9-0331-4a84-9d90-72c17fd990e9" containerID="9653f456152f92425a47c95f785070b0306c9fa97d12fbf6a44a908eaa2650bc" exitCode=0
Jul 24 21:00:01.652952 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:01.652800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst" event={"ID":"09c855c9-0331-4a84-9d90-72c17fd990e9","Type":"ContainerDied","Data":"9653f456152f92425a47c95f785070b0306c9fa97d12fbf6a44a908eaa2650bc"}
Jul 24 21:00:02.805539 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:02.805515    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"
Jul 24 21:00:02.832220 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:02.832192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"
Jul 24 21:00:03.663218 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.663184    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"
Jul 24 21:00:03.663407 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.663225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst" event={"ID":"09c855c9-0331-4a84-9d90-72c17fd990e9","Type":"ContainerDied","Data":"f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c"}
Jul 24 21:00:03.663407 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.663280    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2f4909937ea7ac34f6e7e14a2dd81f986af676783422ec41c8b5d5fe728343c"
Jul 24 21:00:03.664519 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.664496    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"
Jul 24 21:00:03.664645 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.664503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5" event={"ID":"5fc499f7-dbd1-47ac-a99c-59f38523007a","Type":"ContainerDied","Data":"d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299"}
Jul 24 21:00:03.664692 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:03.664644    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d587c51e1ee9f17bb3296fa74b14a7941f1bbde93d760b39580ee4aace11f299"
Jul 24 21:00:09.092096 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:09.092061    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:00:28.202440 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:28.202359    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:00:32.518496 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:32.518460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:00:33.298801 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:33.298769    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 21:00:33.316364 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:33.316281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:00:34.406177 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:34.406147    2577 scope.go:117] "RemoveContainer" containerID="511ed5f4faae8fb4367944862fa9adce095c09a2b43e1a16c213c0ac95340ca6"
Jul 24 21:00:45.488970 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:45.488937    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:00:48.318523 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.318446    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" podUID="44733221-8db1-484a-9df9-ebc53158426c" containerName="main" containerID="cri-o://c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899" gracePeriod=45
Jul 24 21:00:48.576144 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.576088    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 21:00:48.667339 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667312    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667497 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667497 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgkdw\" (UniqueName: \"kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667497 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667414    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667497 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667440    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667738 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667515    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home\") pod \"44733221-8db1-484a-9df9-ebc53158426c\" (UID: \"44733221-8db1-484a-9df9-ebc53158426c\") "
Jul 24 21:00:48.667738 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:00:48.667849 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667821    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache" (OuterVolumeSpecName: "model-cache") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:00:48.667849 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667832    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home" (OuterVolumeSpecName: "home") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:00:48.667944 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.667881    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.669570 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.669547    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 21:00:48.670091 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.670073    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm" (OuterVolumeSpecName: "dshm") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:00:48.670157 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.670086    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw" (OuterVolumeSpecName: "kube-api-access-cgkdw") pod "44733221-8db1-484a-9df9-ebc53158426c" (UID: "44733221-8db1-484a-9df9-ebc53158426c"). InnerVolumeSpecName "kube-api-access-cgkdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:00:48.768856 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.768828    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.768856 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.768851    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.768856 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.768862    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgkdw\" (UniqueName: \"kubernetes.io/projected/44733221-8db1-484a-9df9-ebc53158426c-kube-api-access-cgkdw\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.769059 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.768872    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44733221-8db1-484a-9df9-ebc53158426c-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.769059 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.768881    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44733221-8db1-484a-9df9-ebc53158426c-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:00:48.901794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.901703    2577 generic.go:356] "Generic (PLEG): container finished" podID="44733221-8db1-484a-9df9-ebc53158426c" containerID="c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899" exitCode=0
Jul 24 21:00:48.901794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.901742    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"
Jul 24 21:00:48.901794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.901751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" event={"ID":"44733221-8db1-484a-9df9-ebc53158426c","Type":"ContainerDied","Data":"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"}
Jul 24 21:00:48.901794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.901788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt" event={"ID":"44733221-8db1-484a-9df9-ebc53158426c","Type":"ContainerDied","Data":"21e34b08bb297b1ca931bb771cf739a93c0519ae8bef9f02832eb29ec08579ce"}
Jul 24 21:00:48.902083 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.901804    2577 scope.go:117] "RemoveContainer" containerID="c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"
Jul 24 21:00:48.911614 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.911574    2577 scope.go:117] "RemoveContainer" containerID="c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"
Jul 24 21:00:48.911859 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:00:48.911832    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899\": container with ID starting with c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899 not found: ID does not exist" containerID="c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"
Jul 24 21:00:48.911918 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.911860    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899"} err="failed to get container status \"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899\": rpc error: code = NotFound desc = could not find container \"c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899\": container with ID starting with c9e3d330562ed5474c133419cde2b504f6aecdcc2e397270559672d1acc23899 not found: ID does not exist"
Jul 24 21:00:48.929742 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.929711    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 21:00:48.932965 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:48.932938    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-449ecf/sub-test-model-449ecf-kserve-bb65c45c7-nxtnt"]
Jul 24 21:00:50.152072 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:50.152041    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44733221-8db1-484a-9df9-ebc53158426c" path="/var/lib/kubelet/pods/44733221-8db1-484a-9df9-ebc53158426c/volumes"
Jul 24 21:00:52.104103 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:52.104075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:00:52.251298 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:52.251263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:00:53.695115 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695085    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:00:53.695490 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44733221-8db1-484a-9df9-ebc53158426c" containerName="main"
Jul 24 21:00:53.695490 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695477    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="44733221-8db1-484a-9df9-ebc53158426c" containerName="main"
Jul 24 21:00:53.695563 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695496    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c855c9-0331-4a84-9d90-72c17fd990e9" containerName="cleanup"
Jul 24 21:00:53.695563 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695501    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c855c9-0331-4a84-9d90-72c17fd990e9" containerName="cleanup"
Jul 24 21:00:53.695563 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695511    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc499f7-dbd1-47ac-a99c-59f38523007a" containerName="cleanup"
Jul 24 21:00:53.695563 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695517    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc499f7-dbd1-47ac-a99c-59f38523007a" containerName="cleanup"
Jul 24 21:00:53.695705 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695578    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc499f7-dbd1-47ac-a99c-59f38523007a" containerName="cleanup"
Jul 24 21:00:53.695705 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695585    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="44733221-8db1-484a-9df9-ebc53158426c" containerName="main"
Jul 24 21:00:53.695705 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.695625    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c855c9-0331-4a84-9d90-72c17fd990e9" containerName="cleanup"
Jul 24 21:00:53.700667 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.700642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.704862 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.704839    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f62e6\"/\"openshift-service-ca.crt\""
Jul 24 21:00:53.705003 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.704907    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f62e6\"/\"default-dockercfg-v2ttx\""
Jul 24 21:00:53.705003 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.704976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f62e6\"/\"kube-root-ca.crt\""
Jul 24 21:00:53.705117 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.705072    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f62e6\"/\"rate-test-model-3f62e6-kserve-self-signed-certs\""
Jul 24 21:00:53.707317 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.707293    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:00:53.713933 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.713909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.714051 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.713937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.714051 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.713964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.714051 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.714009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcrl9\" (UniqueName: \"kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.714051 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.714046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.714186 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.714072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.723508 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.723479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:00:53.814799 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.814799 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.814974 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.814974 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcrl9\" (UniqueName: \"kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.814974 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.814974 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.814963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.815297 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.815275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.815375 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.815306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.815375 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.815325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.817238 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.817219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.817383 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.817367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:53.825490 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:53.825469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcrl9\" (UniqueName: \"kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9\") pod \"rate-test-model-3f62e6-kserve-68445648b7-gvn4x\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") " pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:54.011956 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:54.011926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:00:54.163246 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:54.163222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:00:54.935085 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:54.935048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" event={"ID":"f7225afc-8fa6-427c-854b-59739afd9c29","Type":"ContainerStarted","Data":"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"}
Jul 24 21:00:54.935464 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:54.935094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" event={"ID":"f7225afc-8fa6-427c-854b-59739afd9c29","Type":"ContainerStarted","Data":"339e9746b20a74f407ca8b3b4524e3898f291742e1e4b545927afc393987c304"}
Jul 24 21:00:54.953316 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:54.953268    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" podStartSLOduration=1.953253944 podStartE2EDuration="1.953253944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:00:54.950800913 +0000 UTC m=+2721.304378079" watchObservedRunningTime="2026-07-24 21:00:54.953253944 +0000 UTC m=+2721.306831126"
Jul 24 21:00:56.796736 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:00:56.796697    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:01:04.012322 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:04.012279    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:01:04.014656 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:04.012339    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:01:04.025054 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:04.025034    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:01:04.994882 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:04.994853    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:01:13.085483 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:13.085451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:01:16.893384 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:16.893341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:01:46.994808 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:01:46.994767    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:02:01.391877 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:01.391844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:02:25.593361 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:25.593330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:02:29.799218 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:29.799183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:32.062210 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:32.062175    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:02:32.070661 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:32.070626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:33.020548 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:33.020518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:39.891122 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:39.891078    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:02:43.561246 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.561207    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7742-139855872825216"]
Jul 24 21:02:43.564655 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.564630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:43.567266 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.567244    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5zkhj\""
Jul 24 21:02:43.570735 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.570716    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7742-139855872825216"]
Jul 24 21:02:43.693480 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.693446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfbg8\" (UniqueName: \"kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8\") pod \"test-curl-7742-139855872825216\" (UID: \"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22\") " pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:43.794335 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.794304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfbg8\" (UniqueName: \"kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8\") pod \"test-curl-7742-139855872825216\" (UID: \"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22\") " pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:43.802535 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.802512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfbg8\" (UniqueName: \"kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8\") pod \"test-curl-7742-139855872825216\" (UID: \"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22\") " pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:43.875436 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:43.875349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:44.007466 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:44.007440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7742-139855872825216"]
Jul 24 21:02:44.009455 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:02:44.009431    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70a5d5ef_1d3f_4f07_96bf_3b0546b93f22.slice/crio-47442035c936a45c329c0777e3c8ba201f6fa23db59eeacad9b9bbadaa6837ff WatchSource:0}: Error finding container 47442035c936a45c329c0777e3c8ba201f6fa23db59eeacad9b9bbadaa6837ff: Status 404 returned error can't find the container with id 47442035c936a45c329c0777e3c8ba201f6fa23db59eeacad9b9bbadaa6837ff
Jul 24 21:02:44.010799 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:44.010780    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 21:02:44.490923 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:44.490887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7742-139855872825216" event={"ID":"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22","Type":"ContainerStarted","Data":"47442035c936a45c329c0777e3c8ba201f6fa23db59eeacad9b9bbadaa6837ff"}
Jul 24 21:02:45.497964 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:45.497928    2577 generic.go:356] "Generic (PLEG): container finished" podID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" containerID="92eda0a489b8723283148c99e3d061add2298c1623ac5da1d57a8939d3597138" exitCode=0
Jul 24 21:02:45.498359 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:45.497987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7742-139855872825216" event={"ID":"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22","Type":"ContainerDied","Data":"92eda0a489b8723283148c99e3d061add2298c1623ac5da1d57a8939d3597138"}
Jul 24 21:02:46.635103 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.635076    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:46.655757 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.655726    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7742-139855872825216_70a5d5ef-1d3f-4f07-96bf-3b0546b93f22/test-curl-7742-139855872825216/0.log"
Jul 24 21:02:46.681810 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.681782    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7742-139855872825216"]
Jul 24 21:02:46.685205 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.685183    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7742-139855872825216"]
Jul 24 21:02:46.721769 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.721747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfbg8\" (UniqueName: \"kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8\") pod \"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22\" (UID: \"70a5d5ef-1d3f-4f07-96bf-3b0546b93f22\") "
Jul 24 21:02:46.724034 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.724006    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8" (OuterVolumeSpecName: "kube-api-access-jfbg8") pod "70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" (UID: "70a5d5ef-1d3f-4f07-96bf-3b0546b93f22"). InnerVolumeSpecName "kube-api-access-jfbg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:02:46.822674 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:46.822640    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfbg8\" (UniqueName: \"kubernetes.io/projected/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22-kube-api-access-jfbg8\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.082450 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.082351    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" podUID="f7225afc-8fa6-427c-854b-59739afd9c29" containerName="main" containerID="cri-o://98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d" gracePeriod=45
Jul 24 21:02:47.327944 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.327922    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:02:47.428978 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.428885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429142 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429142 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcrl9\" (UniqueName: \"kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429142 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429112    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429331 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429147    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429331 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429194    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir\") pod \"f7225afc-8fa6-427c-854b-59739afd9c29\" (UID: \"f7225afc-8fa6-427c-854b-59739afd9c29\") "
Jul 24 21:02:47.429441 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:02:47.429497 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429432    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home" (OuterVolumeSpecName: "home") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:02:47.429659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.429638    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:02:47.431340 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.431316    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 21:02:47.431442 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.431350    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm" (OuterVolumeSpecName: "dshm") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:02:47.431442 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.431367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9" (OuterVolumeSpecName: "kube-api-access-bcrl9") pod "f7225afc-8fa6-427c-854b-59739afd9c29" (UID: "f7225afc-8fa6-427c-854b-59739afd9c29"). InnerVolumeSpecName "kube-api-access-bcrl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:02:47.507951 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.507922    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:47.508143 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.507967    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47442035c936a45c329c0777e3c8ba201f6fa23db59eeacad9b9bbadaa6837ff"
Jul 24 21:02:47.509332 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.509303    2577 generic.go:356] "Generic (PLEG): container finished" podID="f7225afc-8fa6-427c-854b-59739afd9c29" containerID="98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d" exitCode=0
Jul 24 21:02:47.509458 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.509338    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"
Jul 24 21:02:47.509458 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.509352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" event={"ID":"f7225afc-8fa6-427c-854b-59739afd9c29","Type":"ContainerDied","Data":"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"}
Jul 24 21:02:47.509458 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.509387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x" event={"ID":"f7225afc-8fa6-427c-854b-59739afd9c29","Type":"ContainerDied","Data":"339e9746b20a74f407ca8b3b4524e3898f291742e1e4b545927afc393987c304"}
Jul 24 21:02:47.509458 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.509407    2577 scope.go:117] "RemoveContainer" containerID="98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"
Jul 24 21:02:47.519561 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.519544    2577 scope.go:117] "RemoveContainer" containerID="98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"
Jul 24 21:02:47.519848 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:02:47.519829    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d\": container with ID starting with 98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d not found: ID does not exist" containerID="98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"
Jul 24 21:02:47.519905 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.519856    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d"} err="failed to get container status \"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d\": rpc error: code = NotFound desc = could not find container \"98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d\": container with ID starting with 98352c9b2593af62b2ecdec3a93dbeee54b78e8f03d3ac48dc2434a44471fa6d not found: ID does not exist"
Jul 24 21:02:47.521745 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:02:47.521723    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7742-139855872825216\" is forbidden: User \"system:node:ip-10-0-136-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-110.ec2.internal' and this object" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:47.530457 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530438    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.530457 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530457    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.530621 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530466    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7225afc-8fa6-427c-854b-59739afd9c29-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.530621 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530473    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.530621 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530481    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcrl9\" (UniqueName: \"kubernetes.io/projected/f7225afc-8fa6-427c-854b-59739afd9c29-kube-api-access-bcrl9\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.530621 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.530489    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7225afc-8fa6-427c-854b-59739afd9c29-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:02:47.536244 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.536224    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:02:47.539578 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:47.539556    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3f62e6/rate-test-model-3f62e6-kserve-68445648b7-gvn4x"]
Jul 24 21:02:47.539578 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:02:47.539564    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7742-139855872825216\" is forbidden: User \"system:node:ip-10-0-136-110.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-110.ec2.internal' and this object" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" pod="opendatahub/test-curl-7742-139855872825216"
Jul 24 21:02:48.152619 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:48.152569    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" path="/var/lib/kubelet/pods/70a5d5ef-1d3f-4f07-96bf-3b0546b93f22/volumes"
Jul 24 21:02:48.153049 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:48.153032    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7225afc-8fa6-427c-854b-59739afd9c29" path="/var/lib/kubelet/pods/f7225afc-8fa6-427c-854b-59739afd9c29/volumes"
Jul 24 21:02:53.587301 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:53.587239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:53.899817 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:53.899704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:54.189975 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:54.189863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:57.406498 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:57.406450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:57.487134 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:57.487102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:57.614127 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:57.614081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:57.964330 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:57.964300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:58.160248 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.160221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:58.256583 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.256494    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:02:58.261103 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261076    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:02:58.261510 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261497    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" containerName="test-curl-7742-139855872825216"
Jul 24 21:02:58.261554 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261513    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" containerName="test-curl-7742-139855872825216"
Jul 24 21:02:58.261554 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261528    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7225afc-8fa6-427c-854b-59739afd9c29" containerName="main"
Jul 24 21:02:58.261554 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261537    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7225afc-8fa6-427c-854b-59739afd9c29" containerName="main"
Jul 24 21:02:58.261680 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261618    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7225afc-8fa6-427c-854b-59739afd9c29" containerName="main"
Jul 24 21:02:58.261680 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.261628    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a5d5ef-1d3f-4f07-96bf-3b0546b93f22" containerName="test-curl-7742-139855872825216"
Jul 24 21:02:58.264948 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.264930    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.267630 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.267608    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-787669\""
Jul 24 21:02:58.276719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.276698    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:02:58.434587 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.434555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lqp\" (UniqueName: \"kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.434949 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.434622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.535551 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.535509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lqp\" (UniqueName: \"kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.535739 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.535570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.536271 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.536252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.543754 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.543725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lqp\" (UniqueName: \"kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp\") pod \"payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.576521 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.576498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:58.710997 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:58.710973    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:02:58.712773 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:02:58.712730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071787f5_0966_4365_b989_e032c20824d8.slice/crio-bcd76b2e2cddaebd3915732b82cd398b4a4e08a1a12b3b774c792256342c1f81 WatchSource:0}: Error finding container bcd76b2e2cddaebd3915732b82cd398b4a4e08a1a12b3b774c792256342c1f81: Status 404 returned error can't find the container with id bcd76b2e2cddaebd3915732b82cd398b4a4e08a1a12b3b774c792256342c1f81
Jul 24 21:02:59.568236 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:59.568196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" event={"ID":"071787f5-0966-4365-b989-e032c20824d8","Type":"ContainerStarted","Data":"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"}
Jul 24 21:02:59.568659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:59.568263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:02:59.568659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:59.568283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" event={"ID":"071787f5-0966-4365-b989-e032c20824d8","Type":"ContainerStarted","Data":"bcd76b2e2cddaebd3915732b82cd398b4a4e08a1a12b3b774c792256342c1f81"}
Jul 24 21:02:59.589538 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:02:59.589480    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" podStartSLOduration=1.589465548 podStartE2EDuration="1.589465548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:02:59.587981691 +0000 UTC m=+2845.941558875" watchObservedRunningTime="2026-07-24 21:02:59.589465548 +0000 UTC m=+2845.943042715"
Jul 24 21:03:00.770543 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:00.770503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:00.871536 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:00.871503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:01.832266 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:01.832234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:05.689962 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:05.689917    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:03:10.574975 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:10.574943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:03:12.735957 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:12.735921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:13.053276 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:13.053248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:13.384072 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:13.383978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:16.366548 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:16.366499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:16.464375 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:16.464343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:16.636467 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:16.636388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:17.187002 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.186964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:17.279161 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.279124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:17.313228 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.313197    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:03:17.319228 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.319207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.321862 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.321842    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5df102\""
Jul 24 21:03:17.321957 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.321862    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5df102-dockercfg-ms4d7\""
Jul 24 21:03:17.326743 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.326717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:03:17.348205 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.348179    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:03:17.352137 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.352119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.366707 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.366677    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:03:17.401713 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.401679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg2g2\" (UniqueName: \"kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.401713 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.401714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lsj5\" (UniqueName: \"kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.401920 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.401753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.401920 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.401810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.502531 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.502425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.502531 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.502477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.502793 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.502554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg2g2\" (UniqueName: \"kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.502793 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.502575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lsj5\" (UniqueName: \"kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.503201 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.503180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.503281 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.503261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.513198 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.513169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg2g2\" (UniqueName: \"kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2\") pod \"payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.513835 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.513816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lsj5\" (UniqueName: \"kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5\") pod \"payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.632017 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.631984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:17.662981 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.662951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:17.987986 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:17.987963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:03:17.990124 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:03:17.990098    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aea49e3_e251_446b_8583_754727865a4e.slice/crio-93d387a4639fba6a74e5da5e8f6b9cee5f5eb92267b76adc7ab566401eb5d0b9 WatchSource:0}: Error finding container 93d387a4639fba6a74e5da5e8f6b9cee5f5eb92267b76adc7ab566401eb5d0b9: Status 404 returned error can't find the container with id 93d387a4639fba6a74e5da5e8f6b9cee5f5eb92267b76adc7ab566401eb5d0b9
Jul 24 21:03:18.020621 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.020575    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:03:18.031871 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:03:18.031841    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904dec01_43f7_4763_8f96_a924a18f02a5.slice/crio-735fef4920d889b5bf935c0468b9c0f363270d90defb2c7c2ba7c27cfd6d99bf WatchSource:0}: Error finding container 735fef4920d889b5bf935c0468b9c0f363270d90defb2c7c2ba7c27cfd6d99bf: Status 404 returned error can't find the container with id 735fef4920d889b5bf935c0468b9c0f363270d90defb2c7c2ba7c27cfd6d99bf
Jul 24 21:03:18.665840 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.665796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" event={"ID":"5aea49e3-e251-446b-8583-754727865a4e","Type":"ContainerStarted","Data":"cb661b4348b4210da3476a8370b1a6017d6dfa445d7a30b9e732acb92b1b80b5"}
Jul 24 21:03:18.666299 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.665872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" event={"ID":"5aea49e3-e251-446b-8583-754727865a4e","Type":"ContainerStarted","Data":"93d387a4639fba6a74e5da5e8f6b9cee5f5eb92267b76adc7ab566401eb5d0b9"}
Jul 24 21:03:18.666299 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.665954    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:18.667359 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.667335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" event={"ID":"904dec01-43f7-4763-8f96-a924a18f02a5","Type":"ContainerStarted","Data":"2708462bef303884903713721ec0ab8d1b4575682986956f30c58c9434423592"}
Jul 24 21:03:18.667462 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.667374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" event={"ID":"904dec01-43f7-4763-8f96-a924a18f02a5","Type":"ContainerStarted","Data":"735fef4920d889b5bf935c0468b9c0f363270d90defb2c7c2ba7c27cfd6d99bf"}
Jul 24 21:03:18.667462 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.667399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:18.681759 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.681714    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" podStartSLOduration=1.681700057 podStartE2EDuration="1.681700057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:03:18.680312081 +0000 UTC m=+2865.033889221" watchObservedRunningTime="2026-07-24 21:03:18.681700057 +0000 UTC m=+2865.035277216"
Jul 24 21:03:18.696187 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:18.696144    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" podStartSLOduration=1.696130922 podStartE2EDuration="1.696130922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:03:18.69439924 +0000 UTC m=+2865.047976410" watchObservedRunningTime="2026-07-24 21:03:18.696130922 +0000 UTC m=+2865.049708061"
Jul 24 21:03:19.796085 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:19.796055    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:03:20.877420 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:20.877384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:29.675439 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:29.675339    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:03:29.675879 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:29.675567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:03:38.036271 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:38.036240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:03:38.156588 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:38.156557    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm_5aea49e3-e251-446b-8583-754727865a4e/payload-processing/0.log"
Jul 24 21:03:53.997273 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:53.997234    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:03:54.001799 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.001778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.005676 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.005648    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-787669\"/\"default-dockercfg-plm79\""
Jul 24 21:03:54.006137 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.006113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-787669\"/\"ipp-route-787669-kserve-self-signed-certs\""
Jul 24 21:03:54.006249 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.006162    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-787669\"/\"kube-root-ca.crt\""
Jul 24 21:03:54.006249 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.006172    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-787669\"/\"openshift-service-ca.crt\""
Jul 24 21:03:54.009138 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.009112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:03:54.020844 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.020817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:03:54.039050 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.039173 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039085    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.039173 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.039271 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.039271 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.039352 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.039308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwmcz\" (UniqueName: \"kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.139991 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.139945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.139991 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.139992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140210 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwmcz\" (UniqueName: \"kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140210 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140210 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140210 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140522 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140640 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.140640 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.140611    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.142411 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.142393    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.142740 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.142725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.148030 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.148008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwmcz\" (UniqueName: \"kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz\") pod \"ipp-route-787669-kserve-5f84c6845f-xlztv\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") " pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.312729 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.312696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:03:54.449996 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.449970    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:03:54.452116 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:03:54.452083    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3b60d5a_e020_4fa6_9aef_c12c022a78ac.slice/crio-3fd24875bdcde9664e53ce3f3c439e14a08d9e5b06f6e36bf4fdcf3488402bf4 WatchSource:0}: Error finding container 3fd24875bdcde9664e53ce3f3c439e14a08d9e5b06f6e36bf4fdcf3488402bf4: Status 404 returned error can't find the container with id 3fd24875bdcde9664e53ce3f3c439e14a08d9e5b06f6e36bf4fdcf3488402bf4
Jul 24 21:03:54.864946 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.864906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" event={"ID":"d3b60d5a-e020-4fa6-9aef-c12c022a78ac","Type":"ContainerStarted","Data":"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"}
Jul 24 21:03:54.865121 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.864957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" event={"ID":"d3b60d5a-e020-4fa6-9aef-c12c022a78ac","Type":"ContainerStarted","Data":"3fd24875bdcde9664e53ce3f3c439e14a08d9e5b06f6e36bf4fdcf3488402bf4"}
Jul 24 21:03:54.883504 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:54.883453    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" podStartSLOduration=1.8834379559999999 podStartE2EDuration="1.883437956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:03:54.880911271 +0000 UTC m=+2901.234488413" watchObservedRunningTime="2026-07-24 21:03:54.883437956 +0000 UTC m=+2901.237015096"
Jul 24 21:03:57.698020 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:03:57.697982    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:04:04.313448 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:04.313408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:04:04.313880 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:04.313788    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:04:04.326293 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:04.326269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:04:04.933317 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:04.933290    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:04:34.586263 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:34.586232    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:04:45.745584 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:45.745556    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:04:45.870727 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:45.870697    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm_5aea49e3-e251-446b-8583-754727865a4e/payload-processing/0.log"
Jul 24 21:04:46.152179 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:46.152134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:47.373737 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:47.373701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:49.954864 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:49.954831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.032928 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.032882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.106070 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.106034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.311719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.311679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.526960 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.526916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.630044 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.629958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:04:50.637371 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.637345    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:04:50.641185 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.641166    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.643975 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.643959    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-fd3ebf-dockercfg-8bmpj\""
Jul 24 21:04:50.644171 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.644150    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-fd3ebf\""
Jul 24 21:04:50.651398 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.651377    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:04:50.673096 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.673072    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:04:50.676659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.676642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.686965 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.686939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:04:50.775318 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.775284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.775496 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.775323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.775496 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.775472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8snz4\" (UniqueName: \"kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.775572 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.775526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgshg\" (UniqueName: \"kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.876666 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.876630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8snz4\" (UniqueName: \"kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.876827 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.876697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgshg\" (UniqueName: \"kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.876827 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.876753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.876827 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.876781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.877531 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.877503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.877686 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.877546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.891046 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.890982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8snz4\" (UniqueName: \"kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4\") pod \"payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:50.891046 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.891019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgshg\" (UniqueName: \"kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.952686 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.952652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:50.988101 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:50.988071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:51.114000 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:51.113968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:04:51.115844 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:04:51.115813    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97101201_b5da_4568_b409_b9d495fe2e8f.slice/crio-dc01415201f07c995d2d13f52a693569d2a86d24868b4902a82c8c3ac200045e WatchSource:0}: Error finding container dc01415201f07c995d2d13f52a693569d2a86d24868b4902a82c8c3ac200045e: Status 404 returned error can't find the container with id dc01415201f07c995d2d13f52a693569d2a86d24868b4902a82c8c3ac200045e
Jul 24 21:04:51.148509 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:51.148479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:04:51.154173 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:04:51.154132    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081c5992_ae4e_4905_8a4b_6a5ba445e36f.slice/crio-6c62c2fb792b6bae0048d8974947a893f79f7a52d42d8933cfcc8f0bfa6319f9 WatchSource:0}: Error finding container 6c62c2fb792b6bae0048d8974947a893f79f7a52d42d8933cfcc8f0bfa6319f9: Status 404 returned error can't find the container with id 6c62c2fb792b6bae0048d8974947a893f79f7a52d42d8933cfcc8f0bfa6319f9
Jul 24 21:04:51.156441 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:51.156415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" event={"ID":"97101201-b5da-4568-b409-b9d495fe2e8f","Type":"ContainerStarted","Data":"dc01415201f07c995d2d13f52a693569d2a86d24868b4902a82c8c3ac200045e"}
Jul 24 21:04:52.162257 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.162214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" event={"ID":"97101201-b5da-4568-b409-b9d495fe2e8f","Type":"ContainerStarted","Data":"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"}
Jul 24 21:04:52.162721 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.162367    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:04:52.163718 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.163697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" event={"ID":"081c5992-ae4e-4905-8a4b-6a5ba445e36f","Type":"ContainerStarted","Data":"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"}
Jul 24 21:04:52.163772 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.163735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" event={"ID":"081c5992-ae4e-4905-8a4b-6a5ba445e36f","Type":"ContainerStarted","Data":"6c62c2fb792b6bae0048d8974947a893f79f7a52d42d8933cfcc8f0bfa6319f9"}
Jul 24 21:04:52.163810 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.163777    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:04:52.180755 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.180698    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" podStartSLOduration=2.180685083 podStartE2EDuration="2.180685083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:04:52.179807437 +0000 UTC m=+2958.533384577" watchObservedRunningTime="2026-07-24 21:04:52.180685083 +0000 UTC m=+2958.534262222"
Jul 24 21:04:52.196379 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:52.196335    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" podStartSLOduration=2.196322307 podStartE2EDuration="2.196322307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:04:52.19473307 +0000 UTC m=+2958.548310210" watchObservedRunningTime="2026-07-24 21:04:52.196322307 +0000 UTC m=+2958.549899501"
Jul 24 21:04:53.686029 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:53.685992    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:04:54.459460 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:04:54.459418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:03.170638 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:03.170581    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:05:03.171005 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:03.170773    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:05:16.666790 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.666747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:16.694626 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.694408    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:05:16.694822 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.694750    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" podUID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" containerName="payload-processing" containerID="cri-o://3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5" gracePeriod=30
Jul 24 21:05:16.703027 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.703000    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:05:16.703285 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.703236    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" podUID="97101201-b5da-4568-b409-b9d495fe2e8f" containerName="payload-pre-processing" containerID="cri-o://642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a" gracePeriod=30
Jul 24 21:05:16.737427 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.737401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:16.803532 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.803494    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:16.962341 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.962314    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:05:16.981393 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:16.981363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:05:17.026260 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8snz4\" (UniqueName: \"kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4\") pod \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") "
Jul 24 21:05:17.026449 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026290    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume\") pod \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\" (UID: \"081c5992-ae4e-4905-8a4b-6a5ba445e36f\") "
Jul 24 21:05:17.026449 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026325    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume\") pod \"97101201-b5da-4568-b409-b9d495fe2e8f\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") "
Jul 24 21:05:17.026449 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgshg\" (UniqueName: \"kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg\") pod \"97101201-b5da-4568-b409-b9d495fe2e8f\" (UID: \"97101201-b5da-4568-b409-b9d495fe2e8f\") "
Jul 24 21:05:17.026747 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026720    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97101201-b5da-4568-b409-b9d495fe2e8f" (UID: "97101201-b5da-4568-b409-b9d495fe2e8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:05:17.026886 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.026755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "081c5992-ae4e-4905-8a4b-6a5ba445e36f" (UID: "081c5992-ae4e-4905-8a4b-6a5ba445e36f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:05:17.028570 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.028548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4" (OuterVolumeSpecName: "kube-api-access-8snz4") pod "081c5992-ae4e-4905-8a4b-6a5ba445e36f" (UID: "081c5992-ae4e-4905-8a4b-6a5ba445e36f"). InnerVolumeSpecName "kube-api-access-8snz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:05:17.028955 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.028933    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg" (OuterVolumeSpecName: "kube-api-access-kgshg") pod "97101201-b5da-4568-b409-b9d495fe2e8f" (UID: "97101201-b5da-4568-b409-b9d495fe2e8f"). InnerVolumeSpecName "kube-api-access-kgshg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:05:17.127998 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.127964    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8snz4\" (UniqueName: \"kubernetes.io/projected/081c5992-ae4e-4905-8a4b-6a5ba445e36f-kube-api-access-8snz4\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:17.127998 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.127991    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/081c5992-ae4e-4905-8a4b-6a5ba445e36f-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:17.127998 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.128002    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97101201-b5da-4568-b409-b9d495fe2e8f-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:17.128234 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.128012    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgshg\" (UniqueName: \"kubernetes.io/projected/97101201-b5da-4568-b409-b9d495fe2e8f-kube-api-access-kgshg\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:17.293984 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.293947    2577 generic.go:356] "Generic (PLEG): container finished" podID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" containerID="3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5" exitCode=0
Jul 24 21:05:17.294169 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.293998    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"
Jul 24 21:05:17.294169 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.294036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" event={"ID":"081c5992-ae4e-4905-8a4b-6a5ba445e36f","Type":"ContainerDied","Data":"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"}
Jul 24 21:05:17.294169 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.294092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8" event={"ID":"081c5992-ae4e-4905-8a4b-6a5ba445e36f","Type":"ContainerDied","Data":"6c62c2fb792b6bae0048d8974947a893f79f7a52d42d8933cfcc8f0bfa6319f9"}
Jul 24 21:05:17.294169 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.294118    2577 scope.go:117] "RemoveContainer" containerID="3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"
Jul 24 21:05:17.295312 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.295288    2577 generic.go:356] "Generic (PLEG): container finished" podID="97101201-b5da-4568-b409-b9d495fe2e8f" containerID="642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a" exitCode=0
Jul 24 21:05:17.295434 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.295340    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"
Jul 24 21:05:17.295434 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.295352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" event={"ID":"97101201-b5da-4568-b409-b9d495fe2e8f","Type":"ContainerDied","Data":"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"}
Jul 24 21:05:17.295434 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.295395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk" event={"ID":"97101201-b5da-4568-b409-b9d495fe2e8f","Type":"ContainerDied","Data":"dc01415201f07c995d2d13f52a693569d2a86d24868b4902a82c8c3ac200045e"}
Jul 24 21:05:17.303466 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.303446    2577 scope.go:117] "RemoveContainer" containerID="3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"
Jul 24 21:05:17.303837 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:05:17.303814    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5\": container with ID starting with 3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5 not found: ID does not exist" containerID="3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"
Jul 24 21:05:17.304005 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.303846    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5"} err="failed to get container status \"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5\": rpc error: code = NotFound desc = could not find container \"3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5\": container with ID starting with 3876ba49439329eb6396a03f7c41fbf48f78454b8434be985c3ba78ee3619cd5 not found: ID does not exist"
Jul 24 21:05:17.304005 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.303870    2577 scope.go:117] "RemoveContainer" containerID="642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"
Jul 24 21:05:17.312461 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.312444    2577 scope.go:117] "RemoveContainer" containerID="642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"
Jul 24 21:05:17.312713 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:05:17.312695    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a\": container with ID starting with 642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a not found: ID does not exist" containerID="642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"
Jul 24 21:05:17.312776 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.312720    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a"} err="failed to get container status \"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a\": rpc error: code = NotFound desc = could not find container \"642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a\": container with ID starting with 642369c36a099960284277d15dbd42daa8e64e847537d46f0154faf099f75f8a not found: ID does not exist"
Jul 24 21:05:17.320605 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.320573    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:05:17.327429 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.327408    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3ebf-86b4dbbb4-x9hb8"]
Jul 24 21:05:17.336923 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.336900    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:05:17.340738 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:17.340718    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fd3ebf-7fb86759f6-5f4nk"]
Jul 24 21:05:18.155991 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:18.155959    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" path="/var/lib/kubelet/pods/081c5992-ae4e-4905-8a4b-6a5ba445e36f/volumes"
Jul 24 21:05:18.156365 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:18.156311    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97101201-b5da-4568-b409-b9d495fe2e8f" path="/var/lib/kubelet/pods/97101201-b5da-4568-b409-b9d495fe2e8f/volumes"
Jul 24 21:05:20.490075 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:20.490040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:20.980978 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:20.980948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:21.228490 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:21.228455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:05:26.917329 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:26.917291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:27.051615 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:27.051563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:27.241514 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:27.241435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:27.469470 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:27.469437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:30.679689 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.679643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:30.956566 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.956481    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"]
Jul 24 21:05:30.956930 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.956916    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97101201-b5da-4568-b409-b9d495fe2e8f" containerName="payload-pre-processing"
Jul 24 21:05:30.956930 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.956930    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97101201-b5da-4568-b409-b9d495fe2e8f" containerName="payload-pre-processing"
Jul 24 21:05:30.957015 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.956944    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" containerName="payload-processing"
Jul 24 21:05:30.957015 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.956950    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" containerName="payload-processing"
Jul 24 21:05:30.957082 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.957016    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="97101201-b5da-4568-b409-b9d495fe2e8f" containerName="payload-pre-processing"
Jul 24 21:05:30.957082 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.957033    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="081c5992-ae4e-4905-8a4b-6a5ba445e36f" containerName="payload-processing"
Jul 24 21:05:30.961851 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.961833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:30.965041 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.965018    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 21:05:30.965041 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.965018    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-85rr2\""
Jul 24 21:05:30.969291 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:30.969265    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"]
Jul 24 21:05:31.065884 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:31.065842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc\" (UID: \"9612e440-bd61-4d5c-8e03-79a082dee7d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:31.166431 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:31.166404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc\" (UID: \"9612e440-bd61-4d5c-8e03-79a082dee7d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:31.167080 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:31.167060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc\" (UID: \"9612e440-bd61-4d5c-8e03-79a082dee7d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:31.273893 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:31.273868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:31.408965 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:31.408939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"]
Jul 24 21:05:31.411095 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:05:31.411065    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9612e440_bd61_4d5c_8e03_79a082dee7d0.slice/crio-3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104 WatchSource:0}: Error finding container 3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104: Status 404 returned error can't find the container with id 3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104
Jul 24 21:05:32.376740 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:32.376653    2577 generic.go:356] "Generic (PLEG): container finished" podID="9612e440-bd61-4d5c-8e03-79a082dee7d0" containerID="b440992a9ff8e117f743d08223d75f14e4920b5ea64d980bdf5556de4d0f0df6" exitCode=0
Jul 24 21:05:32.377103 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:32.376739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc" event={"ID":"9612e440-bd61-4d5c-8e03-79a082dee7d0","Type":"ContainerDied","Data":"b440992a9ff8e117f743d08223d75f14e4920b5ea64d980bdf5556de4d0f0df6"}
Jul 24 21:05:32.377103 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:32.376791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc" event={"ID":"9612e440-bd61-4d5c-8e03-79a082dee7d0","Type":"ContainerStarted","Data":"3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104"}
Jul 24 21:05:33.520581 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:33.520559    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:33.590197 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:33.590169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca\") pod \"9612e440-bd61-4d5c-8e03-79a082dee7d0\" (UID: \"9612e440-bd61-4d5c-8e03-79a082dee7d0\") "
Jul 24 21:05:33.590567 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:33.590549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9612e440-bd61-4d5c-8e03-79a082dee7d0" (UID: "9612e440-bd61-4d5c-8e03-79a082dee7d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:05:33.691812 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:33.691728    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9612e440-bd61-4d5c-8e03-79a082dee7d0-maas-api-service-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:34.389734 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:34.389702    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"
Jul 24 21:05:34.389927 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:34.389738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc" event={"ID":"9612e440-bd61-4d5c-8e03-79a082dee7d0","Type":"ContainerDied","Data":"3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104"}
Jul 24 21:05:34.389927 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:34.389808    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e1ff66ab7ec0c63ff662692bdaddf86859b09e19c7b5c0fe22a5f70184d6104"
Jul 24 21:05:45.629834 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:45.629796    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:05:46.638582 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.638542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:46.699865 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.699829    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:05:46.700118 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.700088    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" podUID="071787f5-0966-4365-b989-e032c20824d8" containerName="payload-pre-processing" containerID="cri-o://cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317" gracePeriod=30
Jul 24 21:05:46.761933 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.761905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:46.867047 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.866990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:46.958366 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:46.958343    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:05:47.111323 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.111291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume\") pod \"071787f5-0966-4365-b989-e032c20824d8\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") "
Jul 24 21:05:47.111489 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.111423    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67lqp\" (UniqueName: \"kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp\") pod \"071787f5-0966-4365-b989-e032c20824d8\" (UID: \"071787f5-0966-4365-b989-e032c20824d8\") "
Jul 24 21:05:47.111727 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.111702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "071787f5-0966-4365-b989-e032c20824d8" (UID: "071787f5-0966-4365-b989-e032c20824d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:05:47.113731 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.113710    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp" (OuterVolumeSpecName: "kube-api-access-67lqp") pod "071787f5-0966-4365-b989-e032c20824d8" (UID: "071787f5-0966-4365-b989-e032c20824d8"). InnerVolumeSpecName "kube-api-access-67lqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:05:47.212543 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.212445    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67lqp\" (UniqueName: \"kubernetes.io/projected/071787f5-0966-4365-b989-e032c20824d8-kube-api-access-67lqp\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:47.212543 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.212482    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/071787f5-0966-4365-b989-e032c20824d8-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:05:47.468255 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.468163    2577 generic.go:356] "Generic (PLEG): container finished" podID="071787f5-0966-4365-b989-e032c20824d8" containerID="cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317" exitCode=0
Jul 24 21:05:47.468255 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.468206    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"
Jul 24 21:05:47.468255 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.468236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" event={"ID":"071787f5-0966-4365-b989-e032c20824d8","Type":"ContainerDied","Data":"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"}
Jul 24 21:05:47.468530 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.468285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr" event={"ID":"071787f5-0966-4365-b989-e032c20824d8","Type":"ContainerDied","Data":"bcd76b2e2cddaebd3915732b82cd398b4a4e08a1a12b3b774c792256342c1f81"}
Jul 24 21:05:47.468530 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.468306    2577 scope.go:117] "RemoveContainer" containerID="cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"
Jul 24 21:05:47.477459 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.477440    2577 scope.go:117] "RemoveContainer" containerID="cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"
Jul 24 21:05:47.477757 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:05:47.477735    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317\": container with ID starting with cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317 not found: ID does not exist" containerID="cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"
Jul 24 21:05:47.477824 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.477769    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317"} err="failed to get container status \"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317\": rpc error: code = NotFound desc = could not find container \"cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317\": container with ID starting with cc17c351d48865ce96dcf47d5a1902e568b9eec3e667038550f7a91852d98317 not found: ID does not exist"
Jul 24 21:05:47.494159 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.494133    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:05:47.500010 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:47.499989    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-787669-7d684595bd-5jhjr"]
Jul 24 21:05:48.152394 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:48.152365    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071787f5-0966-4365-b989-e032c20824d8" path="/var/lib/kubelet/pods/071787f5-0966-4365-b989-e032c20824d8/volumes"
Jul 24 21:05:51.110898 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:51.110862    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"]
Jul 24 21:05:51.115181 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:51.115152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:51.116967 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:51.116948    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-787669-9g7mc"]
Jul 24 21:05:51.546428 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:51.546398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:05:52.157335 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:52.157304    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9612e440-bd61-4d5c-8e03-79a082dee7d0" path="/var/lib/kubelet/pods/9612e440-bd61-4d5c-8e03-79a082dee7d0/volumes"
Jul 24 21:05:56.491134 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:56.491093    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:05:56.503531 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:56.503500    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:05:56.556936 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:05:56.556900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:11.511749 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.511705    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" podUID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" containerName="main" containerID="cri-o://2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80" gracePeriod=45
Jul 24 21:06:11.773869 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.773808    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:06:11.846671 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.846861 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846682    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.846861 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwmcz\" (UniqueName: \"kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.846861 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846821    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.846861 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.847073 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846885    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm\") pod \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\" (UID: \"d3b60d5a-e020-4fa6-9aef-c12c022a78ac\") "
Jul 24 21:06:11.847073 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.846925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home" (OuterVolumeSpecName: "home") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:06:11.847073 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.847056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:06:11.847225 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.847200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:06:11.847279 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.847224    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tmp-dir\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:11.847279 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.847246    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-home\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:11.849169 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.849147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 21:06:11.849578 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.849552    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm" (OuterVolumeSpecName: "dshm") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 21:06:11.849578 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.849568    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz" (OuterVolumeSpecName: "kube-api-access-jwmcz") pod "d3b60d5a-e020-4fa6-9aef-c12c022a78ac" (UID: "d3b60d5a-e020-4fa6-9aef-c12c022a78ac"). InnerVolumeSpecName "kube-api-access-jwmcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:06:11.952480 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.948791    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-model-cache\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:11.952480 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.948844    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-dshm\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:11.952480 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.948860    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-tls-certs\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:11.952480 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:11.948884    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwmcz\" (UniqueName: \"kubernetes.io/projected/d3b60d5a-e020-4fa6-9aef-c12c022a78ac-kube-api-access-jwmcz\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:12.594727 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.594693    2577 generic.go:356] "Generic (PLEG): container finished" podID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" containerID="2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80" exitCode=0
Jul 24 21:06:12.595174 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.594759    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"
Jul 24 21:06:12.595174 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.594782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" event={"ID":"d3b60d5a-e020-4fa6-9aef-c12c022a78ac","Type":"ContainerDied","Data":"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"}
Jul 24 21:06:12.595174 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.594827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv" event={"ID":"d3b60d5a-e020-4fa6-9aef-c12c022a78ac","Type":"ContainerDied","Data":"3fd24875bdcde9664e53ce3f3c439e14a08d9e5b06f6e36bf4fdcf3488402bf4"}
Jul 24 21:06:12.595174 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.594844    2577 scope.go:117] "RemoveContainer" containerID="2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"
Jul 24 21:06:12.603900 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.603875    2577 scope.go:117] "RemoveContainer" containerID="2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"
Jul 24 21:06:12.604160 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:06:12.604138    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80\": container with ID starting with 2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80 not found: ID does not exist" containerID="2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"
Jul 24 21:06:12.604227 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.604168    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80"} err="failed to get container status \"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80\": rpc error: code = NotFound desc = could not find container \"2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80\": container with ID starting with 2709753b33ac2686c1cb4108f45f217163f38dfe717bd9c82ed363d9fb017a80 not found: ID does not exist"
Jul 24 21:06:12.612944 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.612923    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:06:12.617776 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:12.617754    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-787669/ipp-route-787669-kserve-5f84c6845f-xlztv"]
Jul 24 21:06:14.152157 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:14.152122    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" path="/var/lib/kubelet/pods/d3b60d5a-e020-4fa6-9aef-c12c022a78ac/volumes"
Jul 24 21:06:29.752713 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:29.752627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:30.406399 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:30.406364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:35.916175 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:35.916144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:41.587951 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:41.587910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:41.684769 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:41.684740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:41.862421 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:41.862345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:42.108153 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:42.108115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:52.627777 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.627742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:52.656522 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.656489    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:06:52.656796 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.656748    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" podUID="5aea49e3-e251-446b-8583-754727865a4e" containerName="payload-processing" containerID="cri-o://cb661b4348b4210da3476a8370b1a6017d6dfa445d7a30b9e732acb92b1b80b5" gracePeriod=30
Jul 24 21:06:52.664424 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.664235    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:06:52.664510 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.664463    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" podUID="904dec01-43f7-4763-8f96-a924a18f02a5" containerName="payload-pre-processing" containerID="cri-o://2708462bef303884903713721ec0ab8d1b4575682986956f30c58c9434423592" gracePeriod=30
Jul 24 21:06:52.694092 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.694067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:52.794375 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.794341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:52.810588 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.810551    2577 generic.go:356] "Generic (PLEG): container finished" podID="904dec01-43f7-4763-8f96-a924a18f02a5" containerID="2708462bef303884903713721ec0ab8d1b4575682986956f30c58c9434423592" exitCode=0
Jul 24 21:06:52.810792 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.810734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" event={"ID":"904dec01-43f7-4763-8f96-a924a18f02a5","Type":"ContainerDied","Data":"2708462bef303884903713721ec0ab8d1b4575682986956f30c58c9434423592"}
Jul 24 21:06:52.812767 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.812743    2577 generic.go:356] "Generic (PLEG): container finished" podID="5aea49e3-e251-446b-8583-754727865a4e" containerID="cb661b4348b4210da3476a8370b1a6017d6dfa445d7a30b9e732acb92b1b80b5" exitCode=0
Jul 24 21:06:52.812868 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.812798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" event={"ID":"5aea49e3-e251-446b-8583-754727865a4e","Type":"ContainerDied","Data":"cb661b4348b4210da3476a8370b1a6017d6dfa445d7a30b9e732acb92b1b80b5"}
Jul 24 21:06:52.925503 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.925480    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:06:52.951224 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:52.951199    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:06:53.032088 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032060    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume\") pod \"5aea49e3-e251-446b-8583-754727865a4e\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") "
Jul 24 21:06:53.032270 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032118    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg2g2\" (UniqueName: \"kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2\") pod \"904dec01-43f7-4763-8f96-a924a18f02a5\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") "
Jul 24 21:06:53.032270 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume\") pod \"904dec01-43f7-4763-8f96-a924a18f02a5\" (UID: \"904dec01-43f7-4763-8f96-a924a18f02a5\") "
Jul 24 21:06:53.032270 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lsj5\" (UniqueName: \"kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5\") pod \"5aea49e3-e251-446b-8583-754727865a4e\" (UID: \"5aea49e3-e251-446b-8583-754727865a4e\") "
Jul 24 21:06:53.032534 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5aea49e3-e251-446b-8583-754727865a4e" (UID: "5aea49e3-e251-446b-8583-754727865a4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:06:53.032708 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.032587    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "904dec01-43f7-4763-8f96-a924a18f02a5" (UID: "904dec01-43f7-4763-8f96-a924a18f02a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:06:53.034420 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.034396    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5" (OuterVolumeSpecName: "kube-api-access-8lsj5") pod "5aea49e3-e251-446b-8583-754727865a4e" (UID: "5aea49e3-e251-446b-8583-754727865a4e"). InnerVolumeSpecName "kube-api-access-8lsj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:06:53.034488 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.034444    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2" (OuterVolumeSpecName: "kube-api-access-hg2g2") pod "904dec01-43f7-4763-8f96-a924a18f02a5" (UID: "904dec01-43f7-4763-8f96-a924a18f02a5"). InnerVolumeSpecName "kube-api-access-hg2g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:06:53.133659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.133624    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5aea49e3-e251-446b-8583-754727865a4e-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:53.133659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.133654    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg2g2\" (UniqueName: \"kubernetes.io/projected/904dec01-43f7-4763-8f96-a924a18f02a5-kube-api-access-hg2g2\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:53.133659 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.133667    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904dec01-43f7-4763-8f96-a924a18f02a5-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:53.133866 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.133678    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lsj5\" (UniqueName: \"kubernetes.io/projected/5aea49e3-e251-446b-8583-754727865a4e-kube-api-access-8lsj5\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:06:53.818869 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.818838    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"
Jul 24 21:06:53.819319 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.818872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm" event={"ID":"5aea49e3-e251-446b-8583-754727865a4e","Type":"ContainerDied","Data":"93d387a4639fba6a74e5da5e8f6b9cee5f5eb92267b76adc7ab566401eb5d0b9"}
Jul 24 21:06:53.819319 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.818934    2577 scope.go:117] "RemoveContainer" containerID="cb661b4348b4210da3476a8370b1a6017d6dfa445d7a30b9e732acb92b1b80b5"
Jul 24 21:06:53.820334 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.820182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8" event={"ID":"904dec01-43f7-4763-8f96-a924a18f02a5","Type":"ContainerDied","Data":"735fef4920d889b5bf935c0468b9c0f363270d90defb2c7c2ba7c27cfd6d99bf"}
Jul 24 21:06:53.820334 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.820211    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"
Jul 24 21:06:53.828084 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.828063    2577 scope.go:117] "RemoveContainer" containerID="2708462bef303884903713721ec0ab8d1b4575682986956f30c58c9434423592"
Jul 24 21:06:53.844470 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.844442    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:06:53.846998 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.846978    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5df102-6748fdddcb-xpwxm"]
Jul 24 21:06:53.856724 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.856704    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:06:53.862983 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:53.862962    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5df102-6d9586d64-7d8c8"]
Jul 24 21:06:54.154560 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:54.154526    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aea49e3-e251-446b-8583-754727865a4e" path="/var/lib/kubelet/pods/5aea49e3-e251-446b-8583-754727865a4e/volumes"
Jul 24 21:06:54.154932 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:54.154918    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904dec01-43f7-4763-8f96-a924a18f02a5" path="/var/lib/kubelet/pods/904dec01-43f7-4763-8f96-a924a18f02a5/volumes"
Jul 24 21:06:55.698758 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:55.698725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:06:57.299769 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:57.299736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:06:57.697044 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:06:57.696971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:03.561939 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:03.561906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:03.659454 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:03.659404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:03.832889 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:03.832812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:04.039804 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:04.039775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:14.687053 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.687020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"]
Jul 24 21:07:14.687639 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.687564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:14.689894 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.689868    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f62e6-29748780-kq6m5"]
Jul 24 21:07:14.714372 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.714342    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 21:07:14.714773 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.714573    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" podUID="c0a000ee-759e-443e-83ac-6ba75c6f0051" containerName="payload-processing" containerID="cri-o://95a6cc3b2595e3c21f386e565653e0df11ee1c762816f4423a65db3cc9c501be" gracePeriod=30
Jul 24 21:07:14.719557 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.719529    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 21:07:14.719798 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.719773    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" podUID="745dd733-36cf-4a35-ae74-18999d369056" containerName="payload-pre-processing" containerID="cri-o://6cd716fe4d0aa0cd4f8ecc8547823eb0c30e149aebcb7e75435a35225541a57e" gracePeriod=30
Jul 24 21:07:14.751303 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.751278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:14.809551 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.809526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:14.926957 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.926926    2577 generic.go:356] "Generic (PLEG): container finished" podID="745dd733-36cf-4a35-ae74-18999d369056" containerID="6cd716fe4d0aa0cd4f8ecc8547823eb0c30e149aebcb7e75435a35225541a57e" exitCode=0
Jul 24 21:07:14.927104 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.926989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" event={"ID":"745dd733-36cf-4a35-ae74-18999d369056","Type":"ContainerDied","Data":"6cd716fe4d0aa0cd4f8ecc8547823eb0c30e149aebcb7e75435a35225541a57e"}
Jul 24 21:07:14.928790 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.928763    2577 generic.go:356] "Generic (PLEG): container finished" podID="c0a000ee-759e-443e-83ac-6ba75c6f0051" containerID="95a6cc3b2595e3c21f386e565653e0df11ee1c762816f4423a65db3cc9c501be" exitCode=0
Jul 24 21:07:14.928903 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:14.928794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" event={"ID":"c0a000ee-759e-443e-83ac-6ba75c6f0051","Type":"ContainerDied","Data":"95a6cc3b2595e3c21f386e565653e0df11ee1c762816f4423a65db3cc9c501be"}
Jul 24 21:07:15.003162 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.003135    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 21:07:15.005259 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.005238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 21:07:15.028437 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.028411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjwzf\" (UniqueName: \"kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf\") pod \"c0a000ee-759e-443e-83ac-6ba75c6f0051\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") "
Jul 24 21:07:15.028627 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.028498    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume\") pod \"c0a000ee-759e-443e-83ac-6ba75c6f0051\" (UID: \"c0a000ee-759e-443e-83ac-6ba75c6f0051\") "
Jul 24 21:07:15.028627 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.028557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpgg4\" (UniqueName: \"kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4\") pod \"745dd733-36cf-4a35-ae74-18999d369056\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") "
Jul 24 21:07:15.028746 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.028628    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume\") pod \"745dd733-36cf-4a35-ae74-18999d369056\" (UID: \"745dd733-36cf-4a35-ae74-18999d369056\") "
Jul 24 21:07:15.028928 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.028899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0a000ee-759e-443e-83ac-6ba75c6f0051" (UID: "c0a000ee-759e-443e-83ac-6ba75c6f0051"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:07:15.029056 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.029036    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "745dd733-36cf-4a35-ae74-18999d369056" (UID: "745dd733-36cf-4a35-ae74-18999d369056"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:07:15.030829 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.030805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4" (OuterVolumeSpecName: "kube-api-access-xpgg4") pod "745dd733-36cf-4a35-ae74-18999d369056" (UID: "745dd733-36cf-4a35-ae74-18999d369056"). InnerVolumeSpecName "kube-api-access-xpgg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:07:15.030929 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.030842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf" (OuterVolumeSpecName: "kube-api-access-qjwzf") pod "c0a000ee-759e-443e-83ac-6ba75c6f0051" (UID: "c0a000ee-759e-443e-83ac-6ba75c6f0051"). InnerVolumeSpecName "kube-api-access-qjwzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:07:15.129990 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.129957    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/745dd733-36cf-4a35-ae74-18999d369056-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:15.129990 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.129985    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjwzf\" (UniqueName: \"kubernetes.io/projected/c0a000ee-759e-443e-83ac-6ba75c6f0051-kube-api-access-qjwzf\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:15.129990 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.129995    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0a000ee-759e-443e-83ac-6ba75c6f0051-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:15.130232 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.130004    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpgg4\" (UniqueName: \"kubernetes.io/projected/745dd733-36cf-4a35-ae74-18999d369056-kube-api-access-xpgg4\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:15.935095 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.935064    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"
Jul 24 21:07:15.935576 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.935067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz" event={"ID":"745dd733-36cf-4a35-ae74-18999d369056","Type":"ContainerDied","Data":"b45e71faa8e9163ea3175158b82e9874ff988b338c46dae12e89cd528b3774ca"}
Jul 24 21:07:15.935576 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.935199    2577 scope.go:117] "RemoveContainer" containerID="6cd716fe4d0aa0cd4f8ecc8547823eb0c30e149aebcb7e75435a35225541a57e"
Jul 24 21:07:15.936487 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.936416    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"
Jul 24 21:07:15.936487 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.936463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5" event={"ID":"c0a000ee-759e-443e-83ac-6ba75c6f0051","Type":"ContainerDied","Data":"867e747fffc8bac21deaf37a1b9020e692b4a4150736070b5ed38f1467f0ff44"}
Jul 24 21:07:15.944976 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.944961    2577 scope.go:117] "RemoveContainer" containerID="95a6cc3b2595e3c21f386e565653e0df11ee1c762816f4423a65db3cc9c501be"
Jul 24 21:07:15.960577 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.960541    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 21:07:15.962779 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.962757    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f62e6-7bc6878696-z5smz"]
Jul 24 21:07:15.973336 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.973309    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 21:07:15.976738 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:15.976717    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f62e6-6c87dc4cf4-v7dm5"]
Jul 24 21:07:16.151616 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:16.151558    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc499f7-dbd1-47ac-a99c-59f38523007a" path="/var/lib/kubelet/pods/5fc499f7-dbd1-47ac-a99c-59f38523007a/volumes"
Jul 24 21:07:16.151955 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:16.151941    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="745dd733-36cf-4a35-ae74-18999d369056" path="/var/lib/kubelet/pods/745dd733-36cf-4a35-ae74-18999d369056/volumes"
Jul 24 21:07:16.152304 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:16.152289    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a000ee-759e-443e-83ac-6ba75c6f0051" path="/var/lib/kubelet/pods/c0a000ee-759e-443e-83ac-6ba75c6f0051/volumes"
Jul 24 21:07:17.987690 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:17.987659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:07:19.250794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:19.250754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:19.645067 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:19.645036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:25.316183 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.316145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:25.377980 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.377947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:25.591616 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.591519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:25.832725 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.832683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:25.943268 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943187    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"]
Jul 24 21:07:25.943609 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943580    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745dd733-36cf-4a35-ae74-18999d369056" containerName="payload-pre-processing"
Jul 24 21:07:25.943650 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943608    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="745dd733-36cf-4a35-ae74-18999d369056" containerName="payload-pre-processing"
Jul 24 21:07:25.943650 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943645    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904dec01-43f7-4763-8f96-a924a18f02a5" containerName="payload-pre-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943653    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="904dec01-43f7-4763-8f96-a924a18f02a5" containerName="payload-pre-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943667    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a000ee-759e-443e-83ac-6ba75c6f0051" containerName="payload-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943672    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a000ee-759e-443e-83ac-6ba75c6f0051" containerName="payload-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943682    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" containerName="main"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943687    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" containerName="main"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943694    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9612e440-bd61-4d5c-8e03-79a082dee7d0" containerName="revoke-keys"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943699    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9612e440-bd61-4d5c-8e03-79a082dee7d0" containerName="revoke-keys"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943709    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071787f5-0966-4365-b989-e032c20824d8" containerName="payload-pre-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943714    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="071787f5-0966-4365-b989-e032c20824d8" containerName="payload-pre-processing"
Jul 24 21:07:25.943719 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943720    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aea49e3-e251-446b-8583-754727865a4e" containerName="payload-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943725    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aea49e3-e251-446b-8583-754727865a4e" containerName="payload-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a000ee-759e-443e-83ac-6ba75c6f0051" containerName="payload-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943788    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="904dec01-43f7-4763-8f96-a924a18f02a5" containerName="payload-pre-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943796    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3b60d5a-e020-4fa6-9aef-c12c022a78ac" containerName="main"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943802    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aea49e3-e251-446b-8583-754727865a4e" containerName="payload-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943810    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="745dd733-36cf-4a35-ae74-18999d369056" containerName="payload-pre-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943814    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="071787f5-0966-4365-b989-e032c20824d8" containerName="payload-pre-processing"
Jul 24 21:07:25.944016 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.943820    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9612e440-bd61-4d5c-8e03-79a082dee7d0" containerName="revoke-keys"
Jul 24 21:07:25.948573 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.948555    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:25.952432 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.952403    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-85rr2\""
Jul 24 21:07:25.952590 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.952409    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 21:07:25.954125 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:25.954103    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"]
Jul 24 21:07:26.029363 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.029324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb\" (UID: \"07d479d3-d26f-4215-9c06-ec41b715ad2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:26.130461 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.130420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb\" (UID: \"07d479d3-d26f-4215-9c06-ec41b715ad2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:26.131125 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.131102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb\" (UID: \"07d479d3-d26f-4215-9c06-ec41b715ad2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:26.261929 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.261898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:26.393943 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.393893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"]
Jul 24 21:07:26.397121 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:07:26.397084    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d479d3_d26f_4215_9c06_ec41b715ad2a.slice/crio-b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c WatchSource:0}: Error finding container b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c: Status 404 returned error can't find the container with id b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c
Jul 24 21:07:26.991092 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.991009    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb" event={"ID":"07d479d3-d26f-4215-9c06-ec41b715ad2a","Type":"ContainerStarted","Data":"d68f4c6b46df63ebf3545e9422c35c6e203420e73572c95b2c4c1a7c8505242a"}
Jul 24 21:07:26.991092 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:26.991069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb" event={"ID":"07d479d3-d26f-4215-9c06-ec41b715ad2a","Type":"ContainerStarted","Data":"b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c"}
Jul 24 21:07:27.007177 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:27.007128    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb" podStartSLOduration=2.007113492 podStartE2EDuration="2.007113492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:07:27.004937611 +0000 UTC m=+3113.358514756" watchObservedRunningTime="2026-07-24 21:07:27.007113492 +0000 UTC m=+3113.360690679"
Jul 24 21:07:27.998049 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:27.998014    2577 generic.go:356] "Generic (PLEG): container finished" podID="07d479d3-d26f-4215-9c06-ec41b715ad2a" containerID="d68f4c6b46df63ebf3545e9422c35c6e203420e73572c95b2c4c1a7c8505242a" exitCode=0
Jul 24 21:07:27.998501 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:27.998074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb" event={"ID":"07d479d3-d26f-4215-9c06-ec41b715ad2a","Type":"ContainerDied","Data":"d68f4c6b46df63ebf3545e9422c35c6e203420e73572c95b2c4c1a7c8505242a"}
Jul 24 21:07:29.133819 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:29.133793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:29.258314 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:29.258281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca\") pod \"07d479d3-d26f-4215-9c06-ec41b715ad2a\" (UID: \"07d479d3-d26f-4215-9c06-ec41b715ad2a\") "
Jul 24 21:07:29.258709 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:29.258684    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07d479d3-d26f-4215-9c06-ec41b715ad2a" (UID: "07d479d3-d26f-4215-9c06-ec41b715ad2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:07:29.359816 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:29.359784    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07d479d3-d26f-4215-9c06-ec41b715ad2a-maas-api-service-ca\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:30.009831 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:30.009798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"
Jul 24 21:07:30.009831 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:30.009819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb" event={"ID":"07d479d3-d26f-4215-9c06-ec41b715ad2a","Type":"ContainerDied","Data":"b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c"}
Jul 24 21:07:30.010062 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:30.009862    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9ff340b1a9c2b9b921f716a6c8e59dec4ebd2de05a53d5b2387b6980502d18c"
Jul 24 21:07:34.497828 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:34.497796    2577 scope.go:117] "RemoveContainer" containerID="60b202ffced74c2a8a682870dbbb0d760bbadb8694571b1bbbab71f898094fb5"
Jul 24 21:07:36.350605 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.350558    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"]
Jul 24 21:07:36.352090 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.352061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:36.353166 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.353145    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-449ecf-29748780-jhkst"]
Jul 24 21:07:36.381894 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.381871    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 21:07:36.382102 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.382076    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" podUID="8752ab26-8e28-4a29-8e28-0b180c0351e2" containerName="payload-processing" containerID="cri-o://fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7" gracePeriod=30
Jul 24 21:07:36.388902 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.388874    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 21:07:36.389375 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.389122    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" podUID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" containerName="payload-pre-processing" containerID="cri-o://5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614" gracePeriod=30
Jul 24 21:07:36.423826 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.423792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:36.518160 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.518110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:36.680173 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.680150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 21:07:36.682965 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.682945    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 21:07:36.824955 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.824920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqskd\" (UniqueName: \"kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd\") pod \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") "
Jul 24 21:07:36.825132 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.824971    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume\") pod \"8752ab26-8e28-4a29-8e28-0b180c0351e2\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") "
Jul 24 21:07:36.825132 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.824997    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4886\" (UniqueName: \"kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886\") pod \"8752ab26-8e28-4a29-8e28-0b180c0351e2\" (UID: \"8752ab26-8e28-4a29-8e28-0b180c0351e2\") "
Jul 24 21:07:36.825132 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.825059    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume\") pod \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\" (UID: \"6e385f26-976a-4c2e-a21e-6cee70ce1df0\") "
Jul 24 21:07:36.825421 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.825312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8752ab26-8e28-4a29-8e28-0b180c0351e2" (UID: "8752ab26-8e28-4a29-8e28-0b180c0351e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:07:36.825533 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.825497    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e385f26-976a-4c2e-a21e-6cee70ce1df0" (UID: "6e385f26-976a-4c2e-a21e-6cee70ce1df0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 21:07:36.827311 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.827289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd" (OuterVolumeSpecName: "kube-api-access-wqskd") pod "6e385f26-976a-4c2e-a21e-6cee70ce1df0" (UID: "6e385f26-976a-4c2e-a21e-6cee70ce1df0"). InnerVolumeSpecName "kube-api-access-wqskd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:07:36.827403 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.827355    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886" (OuterVolumeSpecName: "kube-api-access-l4886") pod "8752ab26-8e28-4a29-8e28-0b180c0351e2" (UID: "8752ab26-8e28-4a29-8e28-0b180c0351e2"). InnerVolumeSpecName "kube-api-access-l4886". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 21:07:36.926344 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.926261    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqskd\" (UniqueName: \"kubernetes.io/projected/6e385f26-976a-4c2e-a21e-6cee70ce1df0-kube-api-access-wqskd\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:36.926344 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.926292    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8752ab26-8e28-4a29-8e28-0b180c0351e2-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:36.926344 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.926301    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4886\" (UniqueName: \"kubernetes.io/projected/8752ab26-8e28-4a29-8e28-0b180c0351e2-kube-api-access-l4886\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:36.926344 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:36.926310    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e385f26-976a-4c2e-a21e-6cee70ce1df0-plugins-config-volume\") on node \"ip-10-0-136-110.ec2.internal\" DevicePath \"\""
Jul 24 21:07:37.046664 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.046626    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" containerID="5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614" exitCode=0
Jul 24 21:07:37.046842 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.046724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" event={"ID":"6e385f26-976a-4c2e-a21e-6cee70ce1df0","Type":"ContainerDied","Data":"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"}
Jul 24 21:07:37.046842 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.046724    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"
Jul 24 21:07:37.046842 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.046815    2577 scope.go:117] "RemoveContainer" containerID="5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"
Jul 24 21:07:37.046983 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.046802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw" event={"ID":"6e385f26-976a-4c2e-a21e-6cee70ce1df0","Type":"ContainerDied","Data":"8e75a139b13c97ca9716231540edb27bba47444ce2048eb8f55f06260b442f77"}
Jul 24 21:07:37.048104 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.048082    2577 generic.go:356] "Generic (PLEG): container finished" podID="8752ab26-8e28-4a29-8e28-0b180c0351e2" containerID="fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7" exitCode=0
Jul 24 21:07:37.048204 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.048115    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"
Jul 24 21:07:37.048433 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.048202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" event={"ID":"8752ab26-8e28-4a29-8e28-0b180c0351e2","Type":"ContainerDied","Data":"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"}
Jul 24 21:07:37.048433 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.048235    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh" event={"ID":"8752ab26-8e28-4a29-8e28-0b180c0351e2","Type":"ContainerDied","Data":"c1c373c2a0c6fdeba7523c3ef959487af46eda95c26b091e5865da6dc30f0e72"}
Jul 24 21:07:37.057195 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.057178    2577 scope.go:117] "RemoveContainer" containerID="5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"
Jul 24 21:07:37.057519 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:07:37.057502    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614\": container with ID starting with 5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614 not found: ID does not exist" containerID="5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"
Jul 24 21:07:37.057564 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.057527    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614"} err="failed to get container status \"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614\": rpc error: code = NotFound desc = could not find container \"5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614\": container with ID starting with 5e66a4ee3a9c4fce0c5251c71f60b899574bff2c1a7b6aff25253635b080e614 not found: ID does not exist"
Jul 24 21:07:37.057564 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.057543    2577 scope.go:117] "RemoveContainer" containerID="fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"
Jul 24 21:07:37.065530 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.065508    2577 scope.go:117] "RemoveContainer" containerID="fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"
Jul 24 21:07:37.065891 ip-10-0-136-110 kubenswrapper[2577]: E0724 21:07:37.065869    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7\": container with ID starting with fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7 not found: ID does not exist" containerID="fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"
Jul 24 21:07:37.066010 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.065897    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7"} err="failed to get container status \"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7\": rpc error: code = NotFound desc = could not find container \"fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7\": container with ID starting with fc129eb1888b593a069e35c45800c92830503d561d8e869c2fb016bcd81aa3f7 not found: ID does not exist"
Jul 24 21:07:37.068444 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.068423    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 21:07:37.075651 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.075628    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-449ecf-d57df55cb-hf5xw"]
Jul 24 21:07:37.084475 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.084450    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 21:07:37.086291 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:37.086269    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-449ecf-6574556f74-fvmgh"]
Jul 24 21:07:38.151662 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:38.151619    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09c855c9-0331-4a84-9d90-72c17fd990e9" path="/var/lib/kubelet/pods/09c855c9-0331-4a84-9d90-72c17fd990e9/volumes"
Jul 24 21:07:38.152044 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:38.151941    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" path="/var/lib/kubelet/pods/6e385f26-976a-4c2e-a21e-6cee70ce1df0/volumes"
Jul 24 21:07:38.152257 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:38.152245    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8752ab26-8e28-4a29-8e28-0b180c0351e2" path="/var/lib/kubelet/pods/8752ab26-8e28-4a29-8e28-0b180c0351e2/volumes"
Jul 24 21:07:39.293580 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:39.293547    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zbrfp"]
Jul 24 21:07:41.016965 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:41.016933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:41.018237 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:41.018211    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"]
Jul 24 21:07:41.020680 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:41.020659    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-449ecf-glgxb"]
Jul 24 21:07:41.411550 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:41.411464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:42.152347 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:42.152305    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d479d3-d26f-4215-9c06-ec41b715ad2a" path="/var/lib/kubelet/pods/07d479d3-d26f-4215-9c06-ec41b715ad2a/volumes"
Jul 24 21:07:47.248734 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:47.248700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:47.311357 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:47.311309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:47.507957 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:47.507872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:47.748309 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:47.748276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:07:54.356736 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:54.356701    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl_08a77acb-41f9-406f-8607-b4f85abfb5c1/copy-manifests/0.log"
Jul 24 21:07:54.364548 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:54.364524    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl_08a77acb-41f9-406f-8607-b4f85abfb5c1/manager/0.log"
Jul 24 21:07:56.946280 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:56.946248    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tfjtb_3f1a5f92-31af-437c-bbb2-b4db6b8318e7/kuadrant-console-plugin/0.log"
Jul 24 21:07:57.307715 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:57.307692    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zbrfp_25b90d72-2337-48bb-90bb-ffd33308f2d8/limitador/0.log"
Jul 24 21:07:58.033065 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:58.033032    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68fdc99b6-nrz9s_f33e003d-1eb1-4235-8869-a9ed5a34f68e/kube-auth-proxy/0.log"
Jul 24 21:07:58.509673 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:58.509645    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:07:59.192910 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:59.192881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9bsth_97391990-abb7-4ec5-b4bd-cd31b4467297/main/0.log"
Jul 24 21:07:59.535198 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:07:59.535166    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-glb2f_f40e76f5-2a6e-428f-a89a-a641a04ed79f/main/0.log"
Jul 24 21:08:05.369454 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369420    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9s798/must-gather-xbnzd"]
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369795    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8752ab26-8e28-4a29-8e28-0b180c0351e2" containerName="payload-processing"
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369812    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8752ab26-8e28-4a29-8e28-0b180c0351e2" containerName="payload-processing"
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369834    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d479d3-d26f-4215-9c06-ec41b715ad2a" containerName="revoke-keys"
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369842    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d479d3-d26f-4215-9c06-ec41b715ad2a" containerName="revoke-keys"
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369852    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" containerName="payload-pre-processing"
Jul 24 21:08:05.369912 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369858    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" containerName="payload-pre-processing"
Jul 24 21:08:05.370342 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369949    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e385f26-976a-4c2e-a21e-6cee70ce1df0" containerName="payload-pre-processing"
Jul 24 21:08:05.370342 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369965    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8752ab26-8e28-4a29-8e28-0b180c0351e2" containerName="payload-processing"
Jul 24 21:08:05.370342 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.369976    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d479d3-d26f-4215-9c06-ec41b715ad2a" containerName="revoke-keys"
Jul 24 21:08:05.373396 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.373380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.375706 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.375682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:08:05.376161 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.376143    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s798\"/\"openshift-service-ca.crt\""
Jul 24 21:08:05.376268 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.376150    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9s798\"/\"kube-root-ca.crt\""
Jul 24 21:08:05.376268 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.376250    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9s798\"/\"default-dockercfg-4d897\""
Jul 24 21:08:05.382825 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.382802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s798/must-gather-xbnzd"]
Jul 24 21:08:05.475225 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.475196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log" limit=1024
Jul 24 21:08:05.481329 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.481305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc5q9\" (UniqueName: \"kubernetes.io/projected/804f2470-611a-4c1b-83cb-a86c5670deca-kube-api-access-qc5q9\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.481447 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.481388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/804f2470-611a-4c1b-83cb-a86c5670deca-must-gather-output\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.582669 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.582634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc5q9\" (UniqueName: \"kubernetes.io/projected/804f2470-611a-4c1b-83cb-a86c5670deca-kube-api-access-qc5q9\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.582827 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.582722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/804f2470-611a-4c1b-83cb-a86c5670deca-must-gather-output\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.583094 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.583076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/804f2470-611a-4c1b-83cb-a86c5670deca-must-gather-output\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.594214 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.594193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc5q9\" (UniqueName: \"kubernetes.io/projected/804f2470-611a-4c1b-83cb-a86c5670deca-kube-api-access-qc5q9\") pod \"must-gather-xbnzd\" (UID: \"804f2470-611a-4c1b-83cb-a86c5670deca\") " pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.683871 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.683791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s798/must-gather-xbnzd"
Jul 24 21:08:05.816546 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.816524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s798/must-gather-xbnzd"]
Jul 24 21:08:05.818564 ip-10-0-136-110 kubenswrapper[2577]: W0724 21:08:05.818533    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod804f2470_611a_4c1b_83cb_a86c5670deca.slice/crio-e8a082f17f8011c8f62e3a9137c6f8dc17a8d37f95a0a205bc22656918ee0d52 WatchSource:0}: Error finding container e8a082f17f8011c8f62e3a9137c6f8dc17a8d37f95a0a205bc22656918ee0d52: Status 404 returned error can't find the container with id e8a082f17f8011c8f62e3a9137c6f8dc17a8d37f95a0a205bc22656918ee0d52
Jul 24 21:08:05.820298 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:05.820282    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 21:08:06.191686 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:06.191650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s798/must-gather-xbnzd" event={"ID":"804f2470-611a-4c1b-83cb-a86c5670deca","Type":"ContainerStarted","Data":"e8a082f17f8011c8f62e3a9137c6f8dc17a8d37f95a0a205bc22656918ee0d52"}
Jul 24 21:08:07.200574 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:07.200530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s798/must-gather-xbnzd" event={"ID":"804f2470-611a-4c1b-83cb-a86c5670deca","Type":"ContainerStarted","Data":"5be57f6f0a9248c294dcb0186e234eaebcc6b1e0548e047361fd6eaaf69412af"}
Jul 24 21:08:07.201087 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:07.200611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s798/must-gather-xbnzd" event={"ID":"804f2470-611a-4c1b-83cb-a86c5670deca","Type":"ContainerStarted","Data":"d842f8831638c626396742397e1cc1f47640c75060591add813c04a46f996e17"}
Jul 24 21:08:07.218650 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:07.218571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9s798/must-gather-xbnzd" podStartSLOduration=1.5061122120000001 podStartE2EDuration="2.218557532s" totalImagesPullingTime="712.44532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 21:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:08:07.216025232 +0000 UTC m=+3153.569602374" watchObservedRunningTime="2026-07-24 21:08:07.218557532 +0000 UTC m=+3153.572134672"
Jul 24 21:08:08.149338 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:08.149291    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nxgsn_502e4008-5e5a-4bee-8936-c5f799313c4f/global-pull-secret-syncer/0.log"
Jul 24 21:08:08.200078 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:08.200045    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n5gpr_6bccd7a2-f090-4038-ab98-1d605c39c415/konnectivity-agent/0.log"
Jul 24 21:08:08.339112 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:08.339083    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-110.ec2.internal_e8d8102a60824d951e2623901e81d027/haproxy/0.log"
Jul 24 21:08:13.640650 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:13.640550    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-tfjtb_3f1a5f92-31af-437c-bbb2-b4db6b8318e7/kuadrant-console-plugin/0.log"
Jul 24 21:08:13.872309 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:13.872286    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zbrfp_25b90d72-2337-48bb-90bb-ffd33308f2d8/limitador/0.log"
Jul 24 21:08:15.533528 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.533460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/alertmanager/0.log"
Jul 24 21:08:15.557489 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.557399    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/config-reloader/0.log"
Jul 24 21:08:15.577797 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.577758    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/kube-rbac-proxy-web/0.log"
Jul 24 21:08:15.601471 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.601445    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/kube-rbac-proxy/0.log"
Jul 24 21:08:15.624778 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.624752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/kube-rbac-proxy-metric/0.log"
Jul 24 21:08:15.650045 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.650001    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/prom-label-proxy/0.log"
Jul 24 21:08:15.673406 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.673370    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_8d963d8f-1885-4f03-b67d-d13cff6dc43c/init-config-reloader/0.log"
Jul 24 21:08:15.743025 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.742991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-k9shx_c90c70df-5a5c-43a2-9bdd-853a87651a08/kube-state-metrics/0.log"
Jul 24 21:08:15.761434 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.761374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-k9shx_c90c70df-5a5c-43a2-9bdd-853a87651a08/kube-rbac-proxy-main/0.log"
Jul 24 21:08:15.784794 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.784716    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-k9shx_c90c70df-5a5c-43a2-9bdd-853a87651a08/kube-rbac-proxy-self/0.log"
Jul 24 21:08:15.811455 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.811424    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-588c96548f-5ffz8_947df3b0-8b1f-404b-a7bd-6fb9d01ba3b5/metrics-server/0.log"
Jul 24 21:08:15.839612 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.839564    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-jk45v_00535512-56a8-4c9e-b62c-5b9e49dc692f/monitoring-plugin/0.log"
Jul 24 21:08:15.961041 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.961010    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjmk_86b45308-92d0-450f-bb8d-4ec2f4dd12cc/node-exporter/0.log"
Jul 24 21:08:15.985188 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:15.985162    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjmk_86b45308-92d0-450f-bb8d-4ec2f4dd12cc/kube-rbac-proxy/0.log"
Jul 24 21:08:16.004915 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.004881    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwjmk_86b45308-92d0-450f-bb8d-4ec2f4dd12cc/init-textfile/0.log"
Jul 24 21:08:16.107949 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.107868    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dvmg2_5bd7b2fa-1edb-4163-8855-967b3ded1df6/kube-rbac-proxy-main/0.log"
Jul 24 21:08:16.133355 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.133311    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dvmg2_5bd7b2fa-1edb-4163-8855-967b3ded1df6/kube-rbac-proxy-self/0.log"
Jul 24 21:08:16.158371 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.158225    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-dvmg2_5bd7b2fa-1edb-4163-8855-967b3ded1df6/openshift-state-metrics/0.log"
Jul 24 21:08:16.204858 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.204829    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/prometheus/0.log"
Jul 24 21:08:16.225121 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.225089    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/config-reloader/0.log"
Jul 24 21:08:16.246823 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.246791    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/thanos-sidecar/0.log"
Jul 24 21:08:16.268276 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.268246    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/kube-rbac-proxy-web/0.log"
Jul 24 21:08:16.289113 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.289079    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/kube-rbac-proxy/0.log"
Jul 24 21:08:16.310872 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.310843    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/kube-rbac-proxy-thanos/0.log"
Jul 24 21:08:16.333656 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.333633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3508d467-ba7a-4b4d-ab1f-5917e8d946b7/init-config-reloader/0.log"
Jul 24 21:08:16.367104 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.367029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-26f5w_bd4cc4db-5d53-4a79-8bfe-8c39d015a7da/prometheus-operator/0.log"
Jul 24 21:08:16.390312 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.390266    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-26f5w_bd4cc4db-5d53-4a79-8bfe-8c39d015a7da/kube-rbac-proxy/0.log"
Jul 24 21:08:16.524082 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.524046    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/thanos-query/0.log"
Jul 24 21:08:16.544267 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.544242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/kube-rbac-proxy-web/0.log"
Jul 24 21:08:16.566320 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.566294    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/kube-rbac-proxy/0.log"
Jul 24 21:08:16.587212 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.587188    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/prom-label-proxy/0.log"
Jul 24 21:08:16.609001 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.608973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/kube-rbac-proxy-rules/0.log"
Jul 24 21:08:16.628339 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:16.628251    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b688554b-ghxn9_7eed69cd-9878-499b-a9b8-6dd5bd754c27/kube-rbac-proxy-metrics/0.log"
Jul 24 21:08:18.094683 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.094642    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"]
Jul 24 21:08:18.102672 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.102641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.106863 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.106837    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"]
Jul 24 21:08:18.217773 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.217731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-lib-modules\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.217981 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.217790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-sys\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.217981 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.217933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-podres\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.218126 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.218106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh7fv\" (UniqueName: \"kubernetes.io/projected/8b9f6735-75ba-44d4-9d38-c1a3335e4775-kube-api-access-nh7fv\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.218283 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.218259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-proc\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.319651 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-lib-modules\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.319850 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-sys\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.319850 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-podres\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.319850 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh7fv\" (UniqueName: \"kubernetes.io/projected/8b9f6735-75ba-44d4-9d38-c1a3335e4775-kube-api-access-nh7fv\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.320010 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-proc\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.320010 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.319966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-proc\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.320122 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.320089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-lib-modules\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.320175 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.320134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-sys\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.320229 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.320197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b9f6735-75ba-44d4-9d38-c1a3335e4775-podres\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.328931 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.328871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh7fv\" (UniqueName: \"kubernetes.io/projected/8b9f6735-75ba-44d4-9d38-c1a3335e4775-kube-api-access-nh7fv\") pod \"perf-node-gather-daemonset-gfmlx\" (UID: \"8b9f6735-75ba-44d4-9d38-c1a3335e4775\") " pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.416889 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.416799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:18.609435 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:18.609348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"]
Jul 24 21:08:19.093996 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:19.093962    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-8pzq7_9005214a-0920-4d14-a92b-c9a7e9b20617/download-server/0.log"
Jul 24 21:08:19.285382 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:19.285342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx" event={"ID":"8b9f6735-75ba-44d4-9d38-c1a3335e4775","Type":"ContainerStarted","Data":"423058ecc5a02a7557b3ad00d2393dd3e3100ed5d3966d28557fdbda15a699f7"}
Jul 24 21:08:19.285886 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:19.285425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx" event={"ID":"8b9f6735-75ba-44d4-9d38-c1a3335e4775","Type":"ContainerStarted","Data":"407bd9cb38a5b8510651d1864c6b85ac2c343cb59ef248412d6b30000860df8c"}
Jul 24 21:08:19.285886 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:19.285547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:19.304228 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:19.304163    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx" podStartSLOduration=1.304141357 podStartE2EDuration="1.304141357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 21:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 21:08:19.301249698 +0000 UTC m=+3165.654826838" watchObservedRunningTime="2026-07-24 21:08:19.304141357 +0000 UTC m=+3165.657718498"
Jul 24 21:08:20.688814 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:20.688780    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxkc_8bc0e3a2-3801-4d91-a2fe-075fa215a0c7/dns/0.log"
Jul 24 21:08:20.709013 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:20.708986    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5qxkc_8bc0e3a2-3801-4d91-a2fe-075fa215a0c7/kube-rbac-proxy/0.log"
Jul 24 21:08:20.825578 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:20.825549    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rcfk_ad07c18d-61bb-43c7-9e8c-08506c9968ad/dns-node-resolver/0.log"
Jul 24 21:08:21.403199 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:21.403168    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9t6cs_a6f0789c-a141-42b5-a35f-5d61f8e2188e/node-ca/0.log"
Jul 24 21:08:22.521739 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:22.521702    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68fdc99b6-nrz9s_f33e003d-1eb1-4235-8869-a9ed5a34f68e/kube-auth-proxy/0.log"
Jul 24 21:08:22.659267 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:22.659231    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:08:23.514831 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:23.514803    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pxsq4_a149787f-5b57-4218-ac7b-bdd0255538d6/serve-healthcheck-canary/0.log"
Jul 24 21:08:24.093676 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:24.093643    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gccc9_f59d6d58-0fc5-4ee4-9239-25b25682cbb4/kube-rbac-proxy/0.log"
Jul 24 21:08:24.114192 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:24.114159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gccc9_f59d6d58-0fc5-4ee4-9239-25b25682cbb4/exporter/0.log"
Jul 24 21:08:24.133985 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:24.133958    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gccc9_f59d6d58-0fc5-4ee4-9239-25b25682cbb4/extractor/0.log"
Jul 24 21:08:25.304129 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:25.304102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9s798/perf-node-gather-daemonset-gfmlx"
Jul 24 21:08:26.834500 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:26.834469    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl_08a77acb-41f9-406f-8607-b4f85abfb5c1/manager/0.log"
Jul 24 21:08:26.854491 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:26.854465    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c48cfcf9b-jvcfl_08a77acb-41f9-406f-8607-b4f85abfb5c1/copy-manifests/0.log"
Jul 24 21:08:28.361346 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:28.361307    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-779f8ddb8c-qbk9s_de498202-8366-4eff-bcc1-3defe8c51a94/manager/0.log"
Jul 24 21:08:28.761506 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:28.761472    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68fdc99b6-nrz9s_f33e003d-1eb1-4235-8869-a9ed5a34f68e/kube-auth-proxy/0.log"
Jul 24 21:08:28.869147 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:28.869121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:08:30.452661 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:30.452573    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68fdc99b6-nrz9s_f33e003d-1eb1-4235-8869-a9ed5a34f68e/kube-auth-proxy/0.log"
Jul 24 21:08:30.560617 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:30.560523    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-g72sw_30801f6e-2fa9-4bfd-b814-0a81cc5c33a2/payload-processing/0.log"
Jul 24 21:08:34.534067 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:34.534039    2577 scope.go:117] "RemoveContainer" containerID="9653f456152f92425a47c95f785070b0306c9fa97d12fbf6a44a908eaa2650bc"
Jul 24 21:08:34.940444 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:34.940371    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/kube-multus-additional-cni-plugins/0.log"
Jul 24 21:08:34.960344 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:34.960322    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/egress-router-binary-copy/0.log"
Jul 24 21:08:34.980264 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:34.980229    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/cni-plugins/0.log"
Jul 24 21:08:35.000458 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.000431    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/bond-cni-plugin/0.log"
Jul 24 21:08:35.019899 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.019876    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/routeoverride-cni/0.log"
Jul 24 21:08:35.042446 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.042421    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/whereabouts-cni-bincopy/0.log"
Jul 24 21:08:35.061760 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.061738    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9px48_48e31ee6-5de8-487f-b301-518d5d7b09c3/whereabouts-cni/0.log"
Jul 24 21:08:35.097508 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.097485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s9b8j_4446fcea-0da3-4c00-aeaf-18a9a01770ad/kube-multus/0.log"
Jul 24 21:08:35.117703 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.117682    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2tlc7_c249c9d6-0606-4b78-9ce7-281e837c2bee/network-metrics-daemon/0.log"
Jul 24 21:08:35.135255 ip-10-0-136-110 kubenswrapper[2577]: I0724 21:08:35.135234    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2tlc7_c249c9d6-0606-4b78-9ce7-281e837c2bee/kube-rbac-proxy/0.log"