Jul 27 21:54:49.707964 ip-10-0-136-91 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 21:54:50.052402 ip-10-0-136-91 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:54:50.052402 ip-10-0-136-91 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 21:54:50.052402 ip-10-0-136-91 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:54:50.052402 ip-10-0-136-91 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 21:54:50.052402 ip-10-0-136-91 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 21:54:50.054015 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.053927    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 21:54:50.058404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058388    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.058404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058403    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058408    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058411    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058414    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058417    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058420    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058423    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058427    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058429    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058432    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058435    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058437    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058440    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058444    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058448    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058451    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058454    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058457    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058460    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058463    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.058472 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058471    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058475    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058478    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058481    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058484    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058488    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058491    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058494    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058496    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058499    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058501    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058504    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058506    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058509    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058511    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058514    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058516    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058519    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058523    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.058953 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058526    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058530    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058532    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058535    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058538    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058540    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058543    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058546    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058549    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058552    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058555    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058558    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058560    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058563    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058566    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058570    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058572    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058575    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058578    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058582    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.059394 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058585    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058588    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058591    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058593    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058596    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058599    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058601    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058604    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058607    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058609    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058611    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058614    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058617    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058619    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058622    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058624    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058627    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058629    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058632    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.059884 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058635    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058637    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058640    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058643    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058646    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058649    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058651    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058654    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058657    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058660    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058663    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058678    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058681    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058683    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058686    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058689    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058692    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058696    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058699    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.060328 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.058701    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060755    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060763    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060766    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060769    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060772    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060776    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060779    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060782    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060785    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060787    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060790    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060793    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.060789 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060796    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060799    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060802    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060804    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060807    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060810    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060812    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060815    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060818    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060821    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060823    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060826    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060830    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060834    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060837    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060840    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060842    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060846    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060848    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.061103 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060851    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060855    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060857    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060860    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060863    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060867    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060870    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060873    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060876    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060878    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060881    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060883    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060886    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060888    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060890    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060894    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060897    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060900    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060903    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060905    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.061754 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060908    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060910    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060913    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060916    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060919    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060922    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060925    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060928    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060931    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060933    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060936    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060938    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060941    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060943    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060946    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060949    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060951    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060954    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060956    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060958    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.062536 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060961    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060964    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060966    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060969    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060972    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060974    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060977    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060980    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060983    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060986    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060988    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060991    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060994    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060996    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.060999    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061001    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061004    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061007    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061011    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061014    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.063112 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061017    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061020    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061022    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061025    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061027    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061029    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061032    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.061034    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063658    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063690    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063696    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063700    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063705    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063710    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063716    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063722    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063729    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063734    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063738    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.063752 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063742    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063746    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063751    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063755    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063760    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063764    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063768    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063773    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063784    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063791    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063798    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063802    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063807    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063810    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063813    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063816    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063818    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063821    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063823    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063826    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.064226 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063829    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063831    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063834    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063837    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063839    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063842    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063845    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063848    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063851    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063854    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063856    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063859    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063861    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063864    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063866    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063869    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063871    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063874    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063877    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063882    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.064763 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063884    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063887    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063889    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063892    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063894    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063898    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063903    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063908    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063911    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063914    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063916    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063919    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063923    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063925    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063928    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063931    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063933    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063936    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063941    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.065238 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063943    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063946    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063949    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063952    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063954    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063957    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063960    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063962    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063965    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063967    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063970    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063973    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063975    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063978    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063981    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063983    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063991    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063994    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063996    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.063999    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.065722 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064002    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064004    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064007    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064009    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064012    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064015    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064017    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064020    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064022    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064103    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064113    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064120    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064125    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064129    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064133    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064137    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064144    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064147    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064151    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064157    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064163    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064168    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 21:54:50.066369 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064172    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064177    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064181    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064186    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064191    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064196    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064199    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064202    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064206    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064209    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064213    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064216    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064219    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064222    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064225    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064229    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064232    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064235    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064239    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064244    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064247    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064250    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064253    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064256    2578 flags.go:64] FLAG: --enable-server="true"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064259    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 21:54:50.066922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064267    2578 flags.go:64] FLAG: --event-burst="100"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064270    2578 flags.go:64] FLAG: --event-qps="50"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064273    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064277    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064280    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064283    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064286    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064289    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064292    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064295    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064297    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064300    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064305    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064308    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064311    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064314    2578 flags.go:64] FLAG: --feature-gates=""
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064318    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064321    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064324    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064327    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064330    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064333    2578 flags.go:64] FLAG: --help="false"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064336    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064339    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 21:54:50.067498 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064342    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064345    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064349    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064353    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064356    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064359    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064362    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064365    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064367    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064371    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064374    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064377    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064379    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064383    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064385    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064388    2578 flags.go:64] FLAG: --lock-file=""
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064391    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064394    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064397    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064403    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064406    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064410    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064413    2578 flags.go:64] FLAG: --logging-format="text"
Jul 27 21:54:50.068104 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064416    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064420    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064423    2578 flags.go:64] FLAG: --manifest-url=""
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064425    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064434    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064437    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064442    2578 flags.go:64] FLAG: --max-pods="110"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064445    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064448    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064451    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064454    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064457    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064460    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064463    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064470    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064473    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064476    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064479    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064482    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064488    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064491    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064494    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064497    2578 flags.go:64] FLAG: --port="10250"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064500    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 21:54:50.068694 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064503    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08b15946c9355317d"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064506    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064509    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064511    2578 flags.go:64] FLAG: --register-node="true"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064514    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064518    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064521    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064524    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064527    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064531    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064534    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064537    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064540    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064543    2578 flags.go:64] FLAG: --runonce="false"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064546    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064549    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064552    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064555    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064557    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064560    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064564    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064567    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064570    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064573    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064575    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064579    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 21:54:50.069284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064582    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064585    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064588    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064594    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064597    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064600    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064604    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064607    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064610    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064612    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064615    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064619    2578 flags.go:64] FLAG: --v="2"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064623    2578 flags.go:64] FLAG: --version="false"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064627    2578 flags.go:64] FLAG: --vmodule=""
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064634    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.064637    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064772    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064776    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064780    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064783    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064786    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064789    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064792    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.069977 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064795    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064797    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064800    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064803    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064805    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064808    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064811    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064813    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064816    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064818    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064821    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064823    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064826    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064829    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064831    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064834    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064837    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064839    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064842    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.070531 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064845    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064851    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064855    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064858    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064861    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064865    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064868    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064871    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064874    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064876    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064879    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064881    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064884    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064887    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064891    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064894    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064897    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064899    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064902    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064905    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.071024 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064907    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064910    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064913    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064915    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064918    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064920    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064923    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064925    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064928    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064930    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064933    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064935    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064938    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064942    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064944    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064947    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064949    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064952    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064955    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.071517 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064958    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064961    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064963    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064966    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064968    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064971    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064973    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064976    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064979    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064981    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064984    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064986    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064989    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064992    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064994    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064997    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.064999    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065002    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065004    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065006    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.071985 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065009    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065012    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065014    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065017    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065019    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065022    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065026    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065028    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065032    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065036    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065039    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065042    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065045    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.065048    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.072466 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.065618    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072123    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072139    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072157    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072163    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072218    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072223    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072226    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072229    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072233    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072236    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072239    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072242    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072244    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072247    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072250    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072252    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072255    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072257    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072259    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.072831 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072262    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072264    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072269    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072273    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072275    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072278    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072280    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072283    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072286    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072288    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072292    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072294    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072297    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072300    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072303    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072306    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072309    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072311    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072314    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.073311 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072317    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072319    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072322    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072325    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072327    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072330    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072333    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072335    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072338    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072341    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072343    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072346    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072348    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072351    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072354    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072357    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072360    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072362    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072364    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072367    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.073918 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072369    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072372    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072374    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072377    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072380    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072382    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072385    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072389    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072391    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072394    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072396    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072399    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072402    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072404    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072407    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072409    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072412    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072414    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072417    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072419    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.074404 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072422    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072425    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072427    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072430    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072432    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072435    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072437    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072440    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072442    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072445    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072447    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072450    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072452    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072455    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072457    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072460    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072462    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072466    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072468    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.074906 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072471    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072476    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072480    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072484    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072487    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072490    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072496    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072607    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072611    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072614    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072617    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072620    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072622    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072625    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072627    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 21:54:50.075356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072630    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072632    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072635    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072637    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072640    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072642    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072644    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072647    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072649    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072652    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072654    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072657    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072659    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072661    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072664    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072681    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072685    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072688    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072692    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 21:54:50.075726 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072695    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072698    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072701    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072705    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072709    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072712    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072715    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072717    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072720    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072723    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072726    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072729    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072731    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072734    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072736    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072739    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072742    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072744    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072747    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072749    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 21:54:50.076189 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072752    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072755    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072758    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072760    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072764    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072767    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072770    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072773    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072776    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072779    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072782    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072785    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072788    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072790    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072793    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072796    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072798    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072801    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072803    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072806    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 21:54:50.076680 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072808    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072811    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072813    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072816    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072818    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072820    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072823    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072826    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072829    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072831    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072833    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072836    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072839    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072842    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072845    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072847    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072850    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072852    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072855    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 21:54:50.077157 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072857    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072860    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072863    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072865    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072869    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072871    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072874    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072876    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072879    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072881    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072884    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072886    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:50.072889    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.072894    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 21:54:50.077622 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.073003    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 21:54:50.078046 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.075005    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 21:54:50.078046 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.076235    2578 server.go:1019] "Starting client certificate rotation"
Jul 27 21:54:50.078046 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.076333    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 21:54:50.078046 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.076363    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 21:54:50.078497 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.078483    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 21:54:50.079681 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.079656    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 21:54:50.092868 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.092845    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 27 21:54:50.094865 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.094849    2578 log.go:25] "Validated CRI v1 image API"
Jul 27 21:54:50.095701 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.095663    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 21:54:50.097965 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.097943    2578 fs.go:135] Filesystem UUIDs: map[214d9744-383b-4412-9882-55db34c6b8f0:/dev/nvme0n1p3 63e776ad-d5df-4ad1-bb10-1782280a07c7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 27 21:54:50.098034 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.097963    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 21:54:50.103968 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.103853    2578 manager.go:216] Machine: {Timestamp:2026-07-27 21:54:50.103347554 +0000 UTC m=+0.290188203 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104498 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ba7a5b990735b480789cc5b977d8a SystemUUID:ec2ba7a5-b990-735b-4807-89cc5b977d8a BootID:efbac559-42cf-4af6-a7d2-f0a22a788265 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7a:bd:88:8f:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7a:bd:88:8f:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:b1:92:aa:e5:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 21:54:50.103968 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.103961    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 21:54:50.104092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.104042    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 21:54:50.104599 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.104583    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 21:54:50.106711 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.106686    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 21:54:50.106910 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.106715    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-91.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 21:54:50.107890 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.107880    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 21:54:50.107923 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.107894    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 21:54:50.107923 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.107901    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 21:54:50.108435 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.108425    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 21:54:50.109083 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.109073    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 21:54:50.110797 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110785    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 21:54:50.110917 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110908    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 21:54:50.110951 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110938    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 21:54:50.110951 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110948    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 21:54:50.111004 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110963    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 21:54:50.111004 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110973    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 27 21:54:50.111004 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.110982    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 21:54:50.111361 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.111349    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 21:54:50.111395 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.111369    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 21:54:50.113426 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.113404    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 21:54:50.114958 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.114945    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 21:54:50.115008 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.114976    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 21:54:50.116865 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116854    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 21:54:50.116899 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116871    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 21:54:50.116899 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116878    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 21:54:50.116899 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116883    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 21:54:50.116899 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116891    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 21:54:50.116899 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116897    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116904    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116909    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116916    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116921    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116937    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116942    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116965    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 21:54:50.117026 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.116970    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 21:54:50.120125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.120113    2578 server.go:1268] "Started kubelet"
Jul 27 21:54:50.120254 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.120196    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 21:54:50.120360 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.120295    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 21:54:50.120420 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.120250    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 21:54:50.121149 ip-10-0-136-91 systemd[1]: Started Kubernetes Kubelet.
Jul 27 21:54:50.121825 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.121724    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 21:54:50.123777 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.123626    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-91.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 21:54:50.123777 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.123660    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-91.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 21:54:50.123777 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.123648    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 21:54:50.124613 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.124597    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 21:54:50.125618 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.125602    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 21:54:50.126791 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.126054    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-91.ec2.internal.18c64433767078c7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-91.ec2.internal,UID:ip-10-0-136-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-91.ec2.internal,},FirstTimestamp:2026-07-27 21:54:50.120083655 +0000 UTC m=+0.306924303,LastTimestamp:2026-07-27 21:54:50.120083655 +0000 UTC m=+0.306924303,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-91.ec2.internal,}"
Jul 27 21:54:50.129214 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.129196    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 21:54:50.129307 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.129217    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 21:54:50.129399 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.129313    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.130341 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130316    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 21:54:50.130558 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130540    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 21:54:50.130606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130561    2578 factory.go:55] Registering systemd factory
Jul 27 21:54:50.130606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130572    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 27 21:54:50.130606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130572    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5v2lx"
Jul 27 21:54:50.130744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130628    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 21:54:50.130744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130641    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 21:54:50.130744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130732    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 21:54:50.130744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130741    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 21:54:50.130861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130811    2578 factory.go:153] Registering CRI-O factory
Jul 27 21:54:50.130861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130825    2578 factory.go:223] Registration of the crio container factory successfully
Jul 27 21:54:50.130861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130847    2578 factory.go:103] Registering Raw factory
Jul 27 21:54:50.130861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.130859    2578 manager.go:1196] Started watching for new ooms in manager
Jul 27 21:54:50.131516 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.131503    2578 manager.go:349] Starting recovery of all containers
Jul 27 21:54:50.133574 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.133545    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-91.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 21:54:50.133753 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.133729    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 21:54:50.141788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.141763    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5v2lx"
Jul 27 21:54:50.144302 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.144185    2578 manager.go:354] Recovery completed
Jul 27 21:54:50.148501 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.148483    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.150267 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150251    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.150324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150280    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.150324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150291    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.150845 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150829    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 21:54:50.150845 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150840    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.150845 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150843    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 21:54:50.150988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.150861    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 21:54:50.151492 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.151474    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.151562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.151501    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.151562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.151514    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.152280 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.152267    2578 policy_none.go:49] "None policy: Start"
Jul 27 21:54:50.152338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.152283    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 21:54:50.152338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.152293    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 21:54:50.153015 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.152953    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-91.ec2.internal.18c64433783d08ad  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-91.ec2.internal,UID:ip-10-0-136-91.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-91.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-91.ec2.internal,},FirstTimestamp:2026-07-27 21:54:50.150267053 +0000 UTC m=+0.337107703,LastTimestamp:2026-07-27 21:54:50.150267053 +0000 UTC m=+0.337107703,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-91.ec2.internal,}"
Jul 27 21:54:50.153259 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.153239    2578 policy_none.go:47] "Start"
Jul 27 21:54:50.193277 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193259    2578 manager.go:337] "Starting Device Plugin manager"
Jul 27 21:54:50.193419 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.193294    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 21:54:50.193419 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193304    2578 server.go:87] "Starting device plugin registration server"
Jul 27 21:54:50.193589 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193575    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 21:54:50.193632 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193594    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 21:54:50.193714 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193695    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 21:54:50.193805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193784    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 21:54:50.193805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.193798    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 21:54:50.194334 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.194317    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 21:54:50.194401 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.194365    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.293914 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.293874    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.294959 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.294794    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.294959 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.294825    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.294959 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.294840    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.294959 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.294866    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.301252 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.301238    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.301298 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.301260    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-91.ec2.internal\": node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.314743 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.314662    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 21:54:50.316019 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.316006    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 21:54:50.316089 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.316038    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 21:54:50.316089 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.316061    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 21:54:50.316180 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.316089    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 21:54:50.318576 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.318560    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:54:50.321569 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.321555    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.416229 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.416174    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"]
Jul 27 21:54:50.416378 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.416271    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.418451 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.418434    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.418544 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.418462    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.418544 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.418472    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.419700 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.419686    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.419803 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.419789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.419841 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.419820    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.420497 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420480    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.420588 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420510    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.420588 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420484    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.420588 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420547    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.420588 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420520    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.420588 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.420564    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.421634 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.421616    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.421688 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.421656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.421724 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.421695    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 21:54:50.422350 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.422335    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 21:54:50.422442 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.422360    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 21:54:50.422442 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.422377    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeHasSufficientPID"
Jul 27 21:54:50.432273 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.432257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.432352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.432280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.432352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.432299    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabb2983197905ab0fef18f09916296b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-91.ec2.internal\" (UID: \"cabb2983197905ab0fef18f09916296b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.449298 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.449279    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-91.ec2.internal\" not found" node="ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.454097 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.454078    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-91.ec2.internal\" not found" node="ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.521784 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.521750    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.532538 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.532647 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.532647 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabb2983197905ab0fef18f09916296b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-91.ec2.internal\" (UID: \"cabb2983197905ab0fef18f09916296b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.532647 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.532647 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532597    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabb2983197905ab0fef18f09916296b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-91.ec2.internal\" (UID: \"cabb2983197905ab0fef18f09916296b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.532647 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.532625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51eb977f3da14d94d0140bfa6a371fad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal\" (UID: \"51eb977f3da14d94d0140bfa6a371fad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.622118 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.622029    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.723017 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.722972    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.751548 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.751519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.757231 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:50.757209    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:50.823917 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.823867    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:50.924461 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:50.924392    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.024999 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.024954    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.075476 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.075449    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 21:54:51.075999 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.075588    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 21:54:51.125515 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.125486    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.146377 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.146332    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 21:49:51 +0000 UTC" deadline="2028-02-23 03:22:10.438815099 +0000 UTC"
Jul 27 21:54:51.146377 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.146371    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13805h27m19.292447603s"
Jul 27 21:54:51.225948 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.225917    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.326334 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.326307    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.326743 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:51.326716    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51eb977f3da14d94d0140bfa6a371fad.slice/crio-692078a1440314281aa21bfd7c902cec682605ef85dc80a3e1dd8a57712eb8aa WatchSource:0}: Error finding container 692078a1440314281aa21bfd7c902cec682605ef85dc80a3e1dd8a57712eb8aa: Status 404 returned error can't find the container with id 692078a1440314281aa21bfd7c902cec682605ef85dc80a3e1dd8a57712eb8aa
Jul 27 21:54:51.330820 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.330801    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:54:51.366572 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.366553    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:54:51.419682 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:51.419636    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcabb2983197905ab0fef18f09916296b.slice/crio-23652692c766ed4ec245c90df1eafe770543e1e1e8fb5c3177bd087ffa9257ac WatchSource:0}: Error finding container 23652692c766ed4ec245c90df1eafe770543e1e1e8fb5c3177bd087ffa9257ac: Status 404 returned error can't find the container with id 23652692c766ed4ec245c90df1eafe770543e1e1e8fb5c3177bd087ffa9257ac
Jul 27 21:54:51.426880 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.426862    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.527649 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:51.527560    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-91.ec2.internal\" not found"
Jul 27 21:54:51.558643 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.558624    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:54:51.572879 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.572857    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:54:51.630218 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.630198    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:51.652493 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.652471    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 21:54:51.653468 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.653455    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal"
Jul 27 21:54:51.675777 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:51.675752    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 21:54:52.112254 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.112218    2578 apiserver.go:52] "Watching apiserver"
Jul 27 21:54:52.122965 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.122925    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 21:54:52.124312 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.124283    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-qgxl7","openshift-network-diagnostics/network-check-target-shbbf","kube-system/konnectivity-agent-c2nbr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal","openshift-multus/multus-additional-cni-plugins-sc5mp","openshift-multus/network-metrics-daemon-4xm4r","openshift-network-operator/iptables-alerter-h4jf5","openshift-ovn-kubernetes/ovnkube-node-wvjqm","kube-system/global-pull-secret-syncer-qlxss","kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72","openshift-cluster-node-tuning-operator/tuned-q75qr","openshift-dns/node-resolver-pxm6b","openshift-image-registry/node-ca-6v88q"]
Jul 27 21:54:52.126985 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.126963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.129640 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.129611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:52.129761 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.129697    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:54:52.129970 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.129948    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 21:54:52.131158 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.131131    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.131258 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.131133    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.131863 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.131843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.132122 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.132104    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wdn2g\""
Jul 27 21:54:52.134675 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.134531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.135284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.135266    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 21:54:52.136169 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.136145    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9pvkt\""
Jul 27 21:54:52.136398 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.136378    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 21:54:52.137132 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.137107    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 21:54:52.137531 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.137446    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.138134 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.138111    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xcrd\""
Jul 27 21:54:52.138134 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.138126    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 21:54:52.138431 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.138408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.138527 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.138426    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.140465 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.140443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.141695 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:52.141813 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef7d343e-8673-4b23-affa-63f4b78fb424-agent-certs\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.141813 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f091a7e-c114-4d02-968a-9cd9f8f404eb-tmp-dir\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.141932 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef7d343e-8673-4b23-affa-63f4b78fb424-konnectivity-ca\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.141932 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f091a7e-c114-4d02-968a-9cd9f8f404eb-hosts-file\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.142033 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.141928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmd2s\" (UniqueName: \"kubernetes.io/projected/9f091a7e-c114-4d02-968a-9cd9f8f404eb-kube-api-access-jmd2s\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.142357 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.142337    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 21:54:52.142617 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.142596    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 21:54:52.145449 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.142815    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x57hn\""
Jul 27 21:54:52.145449 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.143084    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:54:52.145449 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.144988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.148163 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.148140    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.151242 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.151027    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.151348 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.151284    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.151614 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.151591    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:54:52.152666 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152278    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.152666 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152516    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jrdj5\""
Jul 27 21:54:52.152666 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152536    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 21:54:52.152895 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152831    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wvqfm\""
Jul 27 21:54:52.153079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152854    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 21:54:52.153079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.152964    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 21:54:52.153079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.153050    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 21:54:52.153253 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.153098    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.153253 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.153106    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 21:54:52.154100 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.153611    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.154100 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.154040    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.154610 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.154294    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 21:54:52.157225 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.156975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.158902 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.158878    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 21:54:52.159004 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.158957    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.159123 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.159096    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.159230 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.159159    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x5s22\""
Jul 27 21:54:52.159627 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.159608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.159789 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.159716    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.160625 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.160220    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wktfj\""
Jul 27 21:54:52.160625 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.160231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 21:54:52.160625 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.160477    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.165462 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.165446    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:54:52.165776 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.165757    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 21:54:52.165916 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.165896    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2b7dr\""
Jul 27 21:54:52.204345 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.204303    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 21:54:52.205441 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.205417    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnfnf"
Jul 27 21:54:52.218417 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.218400    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnfnf"
Jul 27 21:54:52.231589 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.231562    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 21:54:52.242762 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.242877 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef7d343e-8673-4b23-affa-63f4b78fb424-konnectivity-ca\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.242877 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-netns\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.242877 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87pwb\" (UniqueName: \"kubernetes.io/projected/43159b45-826d-476f-9b4c-694e11a72cef-kube-api-access-87pwb\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.242877 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-lib-modules\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vf4f\" (UniqueName: \"kubernetes.io/projected/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-kube-api-access-2vf4f\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-kubelet\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-systemd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-netd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.242998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-config\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-bin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243092 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243048    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-node-log\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-netns\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-multus-daemon-config\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2md8\" (UniqueName: \"kubernetes.io/projected/4fe713f4-7663-4694-83df-feab1b0a9202-kube-api-access-r2md8\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-system-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-cnibin\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-device-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-system-cni-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.243352 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94335150-3262-4874-a581-23fe3a6130af-iptables-alerter-script\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-systemd-units\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef7d343e-8673-4b23-affa-63f4b78fb424-konnectivity-ca\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-registration-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-sys-fs\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-run\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.243600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243572    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-env-overrides\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-serviceca\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-etc-selinux\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-kubelet-config\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkfb\" (UniqueName: \"kubernetes.io/projected/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-kube-api-access-9fkfb\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-socket-dir-parent\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f091a7e-c114-4d02-968a-9cd9f8f404eb-tmp-dir\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwcbt\" (UniqueName: \"kubernetes.io/projected/8798a1c7-4dca-494d-9ff3-de38f9496c46-kube-api-access-lwcbt\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-os-release\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.243960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.243914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-k8s-cni-cncf-io\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244036    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w66nv\" (UniqueName: \"kubernetes.io/projected/aa83414d-a19d-4577-8fb4-8e209e89c167-kube-api-access-w66nv\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94335150-3262-4874-a581-23fe3a6130af-host-slash\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-cni-binary-copy\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-conf-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-socket-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-sys\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-dbus\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244212    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f091a7e-c114-4d02-968a-9cd9f8f404eb-tmp-dir\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.244265 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-etc-kubernetes\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-os-release\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244310    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysconfig\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-kubernetes\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvh26\" (UniqueName: \"kubernetes.io/projected/94335150-3262-4874-a581-23fe3a6130af-kube-api-access-gvh26\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef7d343e-8673-4b23-affa-63f4b78fb424-agent-certs\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-var-lib-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovn-node-metrics-cert\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.244637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-var-lib-kubelet\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-script-lib\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-host\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-cnibin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-multus\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-conf\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-tmp\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zndvt\" (UniqueName: \"kubernetes.io/projected/230f7314-0b96-4365-95d2-3e270e8d1152-kube-api-access-zndvt\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f091a7e-c114-4d02-968a-9cd9f8f404eb-hosts-file\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244924    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-slash\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244917    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-log-socket\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.244975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-hostroot\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-binary-copy\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-modprobe-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245036    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.245191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-bin\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245071    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-etc-tuned\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmd2s\" (UniqueName: \"kubernetes.io/projected/9f091a7e-c114-4d02-968a-9cd9f8f404eb-kube-api-access-jmd2s\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-etc-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-ovn\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-kubelet\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-multus-certs\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245276    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-systemd\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-host\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.246078 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.245569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f091a7e-c114-4d02-968a-9cd9f8f404eb-hosts-file\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.249542 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.249521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef7d343e-8673-4b23-affa-63f4b78fb424-agent-certs\") pod \"konnectivity-agent-c2nbr\" (UID: \"ef7d343e-8673-4b23-affa-63f4b78fb424\") " pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.256092 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.256072    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:54:52.256180 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.256102    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:54:52.256180 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.256115    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:52.256305 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.256187    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:54:52.756161978 +0000 UTC m=+2.943002618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:52.262539 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.262519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmd2s\" (UniqueName: \"kubernetes.io/projected/9f091a7e-c114-4d02-968a-9cd9f8f404eb-kube-api-access-jmd2s\") pod \"node-resolver-pxm6b\" (UID: \"9f091a7e-c114-4d02-968a-9cd9f8f404eb\") " pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.321959 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.321900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal" event={"ID":"cabb2983197905ab0fef18f09916296b","Type":"ContainerStarted","Data":"23652692c766ed4ec245c90df1eafe770543e1e1e8fb5c3177bd087ffa9257ac"}
Jul 27 21:54:52.323225 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.323191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal" event={"ID":"51eb977f3da14d94d0140bfa6a371fad","Type":"ContainerStarted","Data":"692078a1440314281aa21bfd7c902cec682605ef85dc80a3e1dd8a57712eb8aa"}
Jul 27 21:54:52.346468 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.346676 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-var-lib-kubelet\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.346676 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-var-lib-kubelet\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.346676 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.346595    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.346723    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:54:52.846663645 +0000 UTC m=+3.033504303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-script-lib\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-host\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-cnibin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-multus\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.346907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-conf\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-tmp\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndvt\" (UniqueName: \"kubernetes.io/projected/230f7314-0b96-4365-95d2-3e270e8d1152-kube-api-access-zndvt\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.346979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-slash\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-log-socket\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-hostroot\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-cnibin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-binary-copy\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-modprobe-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-bin\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347179    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-host\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-etc-tuned\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-multus\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-etc-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.347324 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-ovn\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-kubelet\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-multus-certs\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-systemd\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-conf\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-host\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-netns\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87pwb\" (UniqueName: \"kubernetes.io/projected/43159b45-826d-476f-9b4c-694e11a72cef-kube-api-access-87pwb\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-lib-modules\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vf4f\" (UniqueName: \"kubernetes.io/projected/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-kube-api-access-2vf4f\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-kubelet\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-systemd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-netd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-config\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-bin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-node-log\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-netns\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.348239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-multus-daemon-config\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2md8\" (UniqueName: \"kubernetes.io/projected/4fe713f4-7663-4694-83df-feab1b0a9202-kube-api-access-r2md8\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-system-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-cnibin\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-device-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347912    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-system-cni-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94335150-3262-4874-a581-23fe3a6130af-iptables-alerter-script\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-systemd-units\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.347995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-registration-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-sys-fs\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-run\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-env-overrides\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-serviceca\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-etc-selinux\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-binary-copy\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.349208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-kubelet-config\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkfb\" (UniqueName: \"kubernetes.io/projected/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-kube-api-access-9fkfb\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-socket-dir-parent\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-modprobe-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwcbt\" (UniqueName: \"kubernetes.io/projected/8798a1c7-4dca-494d-9ff3-de38f9496c46-kube-api-access-lwcbt\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-os-release\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-bin\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-os-release\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-k8s-cni-cncf-io\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348566    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-kubelet-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-slash\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w66nv\" (UniqueName: \"kubernetes.io/projected/aa83414d-a19d-4577-8fb4-8e209e89c167-kube-api-access-w66nv\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-host\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348629    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94335150-3262-4874-a581-23fe3a6130af-host-slash\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-cni-binary-copy\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350079 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.348712    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-conf-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-socket-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.348784    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:54:52.848765993 +0000 UTC m=+3.035606646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-sys\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-dbus\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-etc-kubernetes\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-script-lib\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-socket-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-os-release\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-etc-kubernetes\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-log-socket\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-os-release\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.348975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysconfig\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-kubernetes\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.350996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvh26\" (UniqueName: \"kubernetes.io/projected/94335150-3262-4874-a581-23fe3a6130af-kube-api-access-gvh26\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-var-lib-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovn-node-metrics-cert\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94335150-3262-4874-a581-23fe3a6130af-host-slash\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-cni-binary-copy\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.349885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-conf-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-hostroot\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-run-netns\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-lib-modules\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-systemd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-cni-netd\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350733    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-registration-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350814    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-sys-fs\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.350873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-run\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-tmp\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovnkube-config\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.351939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-kubelet-config\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8798a1c7-4dca-494d-9ff3-de38f9496c46-env-overrides\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-multus-socket-dir-parent\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-run-ovn\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-etc-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-host-kubelet\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-serviceca\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-etc-selinux\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-cni-bin\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-kubernetes\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-node-log\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-netns\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysconfig\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.351975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-sysctl-d\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352018    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4fe713f4-7663-4694-83df-feab1b0a9202-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352025    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-k8s-cni-cncf-io\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-var-lib-kubelet\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.352880 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-sys\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-host-run-multus-certs\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352167    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aa83414d-a19d-4577-8fb4-8e209e89c167-device-dir\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352208    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6d161cf-b051-4a5c-98f0-ac23f474b98b-dbus\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352211    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/230f7314-0b96-4365-95d2-3e270e8d1152-etc-systemd\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43159b45-826d-476f-9b4c-694e11a72cef-system-cni-dir\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-cnibin\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43159b45-826d-476f-9b4c-694e11a72cef-multus-daemon-config\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-system-cni-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-systemd-units\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4fe713f4-7663-4694-83df-feab1b0a9202-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8798a1c7-4dca-494d-9ff3-de38f9496c46-var-lib-openvswitch\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.352825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94335150-3262-4874-a581-23fe3a6130af-iptables-alerter-script\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.353644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.353199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8798a1c7-4dca-494d-9ff3-de38f9496c46-ovn-node-metrics-cert\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.355010 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.354970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/230f7314-0b96-4365-95d2-3e270e8d1152-etc-tuned\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.371851 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.371792    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w66nv\" (UniqueName: \"kubernetes.io/projected/aa83414d-a19d-4577-8fb4-8e209e89c167-kube-api-access-w66nv\") pod \"aws-ebs-csi-driver-node-prm72\" (UID: \"aa83414d-a19d-4577-8fb4-8e209e89c167\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.373799 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.373772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87pwb\" (UniqueName: \"kubernetes.io/projected/43159b45-826d-476f-9b4c-694e11a72cef-kube-api-access-87pwb\") pod \"multus-qgxl7\" (UID: \"43159b45-826d-476f-9b4c-694e11a72cef\") " pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.374656 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.374633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwcbt\" (UniqueName: \"kubernetes.io/projected/8798a1c7-4dca-494d-9ff3-de38f9496c46-kube-api-access-lwcbt\") pod \"ovnkube-node-wvjqm\" (UID: \"8798a1c7-4dca-494d-9ff3-de38f9496c46\") " pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.375596 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.375571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zndvt\" (UniqueName: \"kubernetes.io/projected/230f7314-0b96-4365-95d2-3e270e8d1152-kube-api-access-zndvt\") pod \"tuned-q75qr\" (UID: \"230f7314-0b96-4365-95d2-3e270e8d1152\") " pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.375991 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.375961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvh26\" (UniqueName: \"kubernetes.io/projected/94335150-3262-4874-a581-23fe3a6130af-kube-api-access-gvh26\") pod \"iptables-alerter-h4jf5\" (UID: \"94335150-3262-4874-a581-23fe3a6130af\") " pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.378590 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.378559    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vf4f\" (UniqueName: \"kubernetes.io/projected/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-kube-api-access-2vf4f\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.383759 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.383727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkfb\" (UniqueName: \"kubernetes.io/projected/75184c67-8fdc-4da3-9aa8-97dcb9f948d8-kube-api-access-9fkfb\") pod \"node-ca-6v88q\" (UID: \"75184c67-8fdc-4da3-9aa8-97dcb9f948d8\") " pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.384313 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.384282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2md8\" (UniqueName: \"kubernetes.io/projected/4fe713f4-7663-4694-83df-feab1b0a9202-kube-api-access-r2md8\") pod \"multus-additional-cni-plugins-sc5mp\" (UID: \"4fe713f4-7663-4694-83df-feab1b0a9202\") " pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.443499 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.443410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pxm6b"
Jul 27 21:54:52.452336 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.452309    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:54:52.461761 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.461729    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qgxl7"
Jul 27 21:54:52.470827 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.470806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sc5mp"
Jul 27 21:54:52.479527 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.479506    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h4jf5"
Jul 27 21:54:52.488779 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.488759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:54:52.504298 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.504273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6v88q"
Jul 27 21:54:52.514654 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.514633    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72"
Jul 27 21:54:52.522875 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.522856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q75qr"
Jul 27 21:54:52.853875 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.853782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:52.853875 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.853848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:52.853891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.853942    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854012    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854010    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:54:53.853991286 +0000 UTC m=+4.040831928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854028    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:54:52.854071 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854036    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:52.854323 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854101    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:54:53.854082134 +0000 UTC m=+4.040922785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:52.854323 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854040    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:52.854323 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:52.854164    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:54:53.854154524 +0000 UTC m=+4.040995167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:53.222823 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:53.222720    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 21:49:53 +0000 UTC" deadline="2028-01-30 01:56:58.034875877 +0000 UTC"
Jul 27 21:54:53.222823 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:53.222759    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13228h2m4.812122033s"
Jul 27 21:54:53.528529 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:53.528501    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa83414d_a19d_4577_8fb4_8e209e89c167.slice/crio-801804d744710aeddf9cee120e294f2acac4e42a03f86ea4471825a097f6043f WatchSource:0}: Error finding container 801804d744710aeddf9cee120e294f2acac4e42a03f86ea4471825a097f6043f: Status 404 returned error can't find the container with id 801804d744710aeddf9cee120e294f2acac4e42a03f86ea4471825a097f6043f
Jul 27 21:54:53.529333 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:53.529298    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230f7314_0b96_4365_95d2_3e270e8d1152.slice/crio-3fce3dc19f03d2a401db16daef0ac93079f606fac5ae47ef5d0dfc494f4f73ef WatchSource:0}: Error finding container 3fce3dc19f03d2a401db16daef0ac93079f606fac5ae47ef5d0dfc494f4f73ef: Status 404 returned error can't find the container with id 3fce3dc19f03d2a401db16daef0ac93079f606fac5ae47ef5d0dfc494f4f73ef
Jul 27 21:54:53.532558 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:53.532531    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94335150_3262_4874_a581_23fe3a6130af.slice/crio-3e8d52552188432120f3b0b3354eaad0f927a2119b73e0f23a364a4f21225c60 WatchSource:0}: Error finding container 3e8d52552188432120f3b0b3354eaad0f927a2119b73e0f23a364a4f21225c60: Status 404 returned error can't find the container with id 3e8d52552188432120f3b0b3354eaad0f927a2119b73e0f23a364a4f21225c60
Jul 27 21:54:53.536362 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:54:53.532784    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8798a1c7_4dca_494d_9ff3_de38f9496c46.slice/crio-651cfee8d892346742dc6621abd7cb958c614a9dad4e251b15e2f8023e5b5c26 WatchSource:0}: Error finding container 651cfee8d892346742dc6621abd7cb958c614a9dad4e251b15e2f8023e5b5c26: Status 404 returned error can't find the container with id 651cfee8d892346742dc6621abd7cb958c614a9dad4e251b15e2f8023e5b5c26
Jul 27 21:54:53.863129 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:53.863089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:53.863271 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:53.863149    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:53.863271 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:53.863200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:53.863341 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863280    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:53.863341 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863324    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:53.863341 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863325    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:54:53.863433 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863356    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:54:55.863334354 +0000 UTC m=+6.050174996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:53.863433 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863349    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:54:53.863433 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863397    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:53.863540 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863449    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:54:55.863435841 +0000 UTC m=+6.050276484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:53.863540 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:53.863468    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:54:55.863458688 +0000 UTC m=+6.050299333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:54.223087 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.222951    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 21:49:53 +0000 UTC" deadline="2028-01-24 01:55:27.178110035 +0000 UTC"
Jul 27 21:54:54.223087 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.223022    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13084h0m32.955091895s"
Jul 27 21:54:54.319250 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.319214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:54.319426 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:54.319364    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:54:54.319828 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.319806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:54.319987 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:54.319907    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:54:54.319987 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.319976    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:54.320147 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:54.320067    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:54:54.340807 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.340771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgxl7" event={"ID":"43159b45-826d-476f-9b4c-694e11a72cef","Type":"ContainerStarted","Data":"2b95e1b06a150285e8d7dcc8fbda5678316fd5a856483e182c84fe5211b4b3b5"}
Jul 27 21:54:54.351071 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.351007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pxm6b" event={"ID":"9f091a7e-c114-4d02-968a-9cd9f8f404eb","Type":"ContainerStarted","Data":"79e56e3e7ce2ae158bb85c9fdc11f8275c378e767d7e31d1b202713bd90eb1d2"}
Jul 27 21:54:54.356718 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.356687    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6v88q" event={"ID":"75184c67-8fdc-4da3-9aa8-97dcb9f948d8","Type":"ContainerStarted","Data":"65aedf5fe47e6155919b7d7f6638a1100dbfb103bc48aec2644f1eeadf36860b"}
Jul 27 21:54:54.362054 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.362024    2578 generic.go:356] "Generic (PLEG): container finished" podID="51eb977f3da14d94d0140bfa6a371fad" containerID="0b2e3ed05496337f09bd256b7ee9c2cd520882dad618fea535e17ef368bc5010" exitCode=0
Jul 27 21:54:54.362160 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.362110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal" event={"ID":"51eb977f3da14d94d0140bfa6a371fad","Type":"ContainerDied","Data":"0b2e3ed05496337f09bd256b7ee9c2cd520882dad618fea535e17ef368bc5010"}
Jul 27 21:54:54.365259 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.365217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerStarted","Data":"be20e4dedf9a2c69ce3a9fa8c001cc6d1870957f2bce9f71eff58b91f9336093"}
Jul 27 21:54:54.375360 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.375263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"651cfee8d892346742dc6621abd7cb958c614a9dad4e251b15e2f8023e5b5c26"}
Jul 27 21:54:54.395345 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.395310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q75qr" event={"ID":"230f7314-0b96-4365-95d2-3e270e8d1152","Type":"ContainerStarted","Data":"3fce3dc19f03d2a401db16daef0ac93079f606fac5ae47ef5d0dfc494f4f73ef"}
Jul 27 21:54:54.418151 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.418112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h4jf5" event={"ID":"94335150-3262-4874-a581-23fe3a6130af","Type":"ContainerStarted","Data":"3e8d52552188432120f3b0b3354eaad0f927a2119b73e0f23a364a4f21225c60"}
Jul 27 21:54:54.430389 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.430349    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72" event={"ID":"aa83414d-a19d-4577-8fb4-8e209e89c167","Type":"ContainerStarted","Data":"801804d744710aeddf9cee120e294f2acac4e42a03f86ea4471825a097f6043f"}
Jul 27 21:54:54.455190 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.455149    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2nbr" event={"ID":"ef7d343e-8673-4b23-affa-63f4b78fb424","Type":"ContainerStarted","Data":"57c7e569b98fd41ac4afba1a06d82f344074319a43d75d27cadb8aec067fed5d"}
Jul 27 21:54:54.466595 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:54.466556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal" event={"ID":"cabb2983197905ab0fef18f09916296b","Type":"ContainerStarted","Data":"4cee690c26e462e845f97d3248110170d1627dd38fffe5665bda21307ca04658"}
Jul 27 21:54:55.502278 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.502238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal" event={"ID":"51eb977f3da14d94d0140bfa6a371fad","Type":"ContainerStarted","Data":"7cc817d54932094bc34cad1a208b8d64e3b3224f0b7abec4cb591a8b34c741a1"}
Jul 27 21:54:55.532197 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.532138    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-91.ec2.internal" podStartSLOduration=3.532114547 podStartE2EDuration="3.532114547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:54:54.494769073 +0000 UTC m=+4.681609732" watchObservedRunningTime="2026-07-27 21:54:55.532114547 +0000 UTC m=+5.718955214"
Jul 27 21:54:55.533325 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.532746    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-91.ec2.internal" podStartSLOduration=3.532737637 podStartE2EDuration="3.532737637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:54:55.531891534 +0000 UTC m=+5.718732197" watchObservedRunningTime="2026-07-27 21:54:55.532737637 +0000 UTC m=+5.719578300"
Jul 27 21:54:55.882432 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.882334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:55.882432 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.882404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:55.882682 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:55.882444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:55.882682 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.882585    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:55.882682 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.882649    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:54:59.882629464 +0000 UTC m=+10.069470131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:55.883129 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883106    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:54:55.883129 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883133    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:54:55.883304 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883147    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:55.883304 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883204    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:54:59.883180638 +0000 UTC m=+10.070021292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:55.883304 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883270    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:55.883476 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:55.883309    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:54:59.883297436 +0000 UTC m=+10.070138078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:56.322029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:56.322085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:56.322334    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:56.322524    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:56.322601    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:56.326737 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:56.322731    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:54:58.319609 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:58.319578    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:58.320047 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:58.319700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:54:58.320094 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:58.320064    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:58.320171 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:58.320149    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:54:58.320239 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:58.320225    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:58.320333 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:58.320313    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:54:59.921233 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:59.921190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:59.921256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:54:59.921284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921381    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921411    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921427    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921438    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921462    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:55:07.921438063 +0000 UTC m=+18.108278707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921510    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:55:07.921473071 +0000 UTC m=+18.108313714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921524    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:54:59.921733 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:54:59.921557    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:07.921544537 +0000 UTC m=+18.108385177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:00.317477    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:00.317504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:00.317527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:00.317633    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:00.317791    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:00.318039 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:00.317885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:02.319496 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:02.319463    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:02.319963 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:02.319573    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:02.319963 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:02.319938    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:02.320110 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:02.320040    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:02.320110 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:02.320109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:02.320215 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:02.320196    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:04.317344 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:04.317308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:04.317344 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:04.317344    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:04.317871 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:04.317308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:04.317871 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:04.317431    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:04.317871 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:04.317507    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:04.317871 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:04.317636    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:06.317050 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:06.317010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:06.317501 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:06.317010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:06.317501 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:06.317162    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:06.317501 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:06.317239    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:06.317501 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:06.317010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:06.317501 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:06.317335    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:07.983559 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:07.983518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:07.983559 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:07.983567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:07.983598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983706    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983769    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983713    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983792    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983809    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983794    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:23.983772932 +0000 UTC m=+34.170613576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983867    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:55:23.983844956 +0000 UTC m=+34.170685600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:55:07.984077 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:07.983892    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:55:23.983875471 +0000 UTC m=+34.170716113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:08.317181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:08.317181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:08.317327    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:08.317195    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:08.317398    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:08.317746 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:08.317524    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:10.317120 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:10.317089    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:10.317573 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:10.317124    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:10.317573 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:10.317182    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:10.317573 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:10.317240    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:10.317573 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:10.317268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:10.317573 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:10.317334    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:12.319898 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:12.319870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:12.319898 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:12.319903    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:12.320332 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:12.319975    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:12.320332 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:12.320237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:12.320332 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:12.320306    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:12.320431 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:12.320367    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:13.543611 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.543200    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6v88q" event={"ID":"75184c67-8fdc-4da3-9aa8-97dcb9f948d8","Type":"ContainerStarted","Data":"eca6b9db74949166dc3b4dd0a0e387a7c0e3ee185b8f6940703677cff01c6406"}
Jul 27 21:55:13.545463 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.545430    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="c73b310cb4e73409ca80d531dab68f5244ef1c3456340893342f81cfbf5b8959" exitCode=0
Jul 27 21:55:13.545737 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.545711    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"c73b310cb4e73409ca80d531dab68f5244ef1c3456340893342f81cfbf5b8959"}
Jul 27 21:55:13.549258 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"a6201f19fcfd51af980d1f84b3196f1066e7868369c1bc3892363a5a2d34423d"}
Jul 27 21:55:13.549359 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549298    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"2e3163cff16fa29ea55cef4a7b62d075ba930a7f4e8052f61ee5649d112aaa28"}
Jul 27 21:55:13.549359 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"fd117a1fdbb6679e3e318b5474347ade1f39c7563e417408c643be990f205348"}
Jul 27 21:55:13.549473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"c614ffc87cbd0acd1dfad0f854ea7b314998ac6ce9d7b9f6190c21be4a6599ba"}
Jul 27 21:55:13.549473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549407    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"7d1e8c693b6ac14178abc2c322c1a06d7d592b86c7b21d2ace34af4fa5fdf014"}
Jul 27 21:55:13.549473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.549451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"8947b30f863fc4580ecb131d75fe13fff78a3da507da16c7def3bf76317c9809"}
Jul 27 21:55:13.550985 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.550963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q75qr" event={"ID":"230f7314-0b96-4365-95d2-3e270e8d1152","Type":"ContainerStarted","Data":"ad6aa5caa435c96f28f2210c6b1191378f54acbf556d445fba0cb00f56d7aa28"}
Jul 27 21:55:13.552583 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.552560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72" event={"ID":"aa83414d-a19d-4577-8fb4-8e209e89c167","Type":"ContainerStarted","Data":"c5f7db89e393fa0b7b6bb34965c023927b631872e7195041120886f01633fdd6"}
Jul 27 21:55:13.554147 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.554129    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2nbr" event={"ID":"ef7d343e-8673-4b23-affa-63f4b78fb424","Type":"ContainerStarted","Data":"002f1242a5961ecf0f893036ea891cc586623c80a85b4ad709c266e7ea68b9e0"}
Jul 27 21:55:13.555733 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.555711    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qgxl7" event={"ID":"43159b45-826d-476f-9b4c-694e11a72cef","Type":"ContainerStarted","Data":"42cf5b424c728b3609ffed457441187ff23a53afc62c8c98424a45f1b534f2df"}
Jul 27 21:55:13.558316 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.558075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pxm6b" event={"ID":"9f091a7e-c114-4d02-968a-9cd9f8f404eb","Type":"ContainerStarted","Data":"19b324e20aa00b434ae8293ef5b23b1c5d630034ee2acae61e9bb46b8292b29f"}
Jul 27 21:55:13.559358 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.559322    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6v88q" podStartSLOduration=3.3795136599999998 podStartE2EDuration="22.559311462s" totalImagesPullingTime="19.179797802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:13.558792628 +0000 UTC m=+23.745633292" watchObservedRunningTime="2026-07-27 21:55:13.559311462 +0000 UTC m=+23.746152121"
Jul 27 21:55:13.576940 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.576890    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qgxl7" podStartSLOduration=3.343614875 podStartE2EDuration="22.57687639s" totalImagesPullingTime="19.233261515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:13.576809447 +0000 UTC m=+23.763650108" watchObservedRunningTime="2026-07-27 21:55:13.57687639 +0000 UTC m=+23.763717049"
Jul 27 21:55:13.597908 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.597864    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q75qr" podStartSLOduration=3.399680091 podStartE2EDuration="22.597843923s" totalImagesPullingTime="19.198163832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:13.597716424 +0000 UTC m=+23.784557084" watchObservedRunningTime="2026-07-27 21:55:13.597843923 +0000 UTC m=+23.784684582"
Jul 27 21:55:13.613908 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.613860    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c2nbr" podStartSLOduration=11.396130312 podStartE2EDuration="22.613843363s" totalImagesPullingTime="11.217713051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:13.613841143 +0000 UTC m=+23.800681801" watchObservedRunningTime="2026-07-27 21:55:13.613843363 +0000 UTC m=+23.800684004"
Jul 27 21:55:13.665196 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.665135    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pxm6b" podStartSLOduration=3.479856362 podStartE2EDuration="22.66511522s" totalImagesPullingTime="19.185258858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:13.663520113 +0000 UTC m=+23.850360777" watchObservedRunningTime="2026-07-27 21:55:13.66511522 +0000 UTC m=+23.851955883"
Jul 27 21:55:13.834420 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:13.834400    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 21:55:14.207612 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.207500    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T21:55:13.834416239Z","UUID":"5fb5ec22-d946-41e4-917d-4f095774d0ed","Handler":null,"Name":"","Endpoint":""}
Jul 27 21:55:14.209135 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.209105    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 21:55:14.209261 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.209148    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 21:55:14.316778 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.316742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:14.316778 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.316756    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:14.317017 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.316879    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:14.317017 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:14.316880    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:14.317017 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:14.316991    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:14.317167 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:14.317091    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:14.563902 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.563189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h4jf5" event={"ID":"94335150-3262-4874-a581-23fe3a6130af","Type":"ContainerStarted","Data":"973922dab9baa5eb6b4043af24e66a476ef8faad4c9b6cea1d05cbe319518ce4"}
Jul 27 21:55:14.565568 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.565539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72" event={"ID":"aa83414d-a19d-4577-8fb4-8e209e89c167","Type":"ContainerStarted","Data":"d661872a2d96edf2a1d50fa0d0726835ea9fdbcfdf6335664ba5eae095d41e9a"}
Jul 27 21:55:14.596079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:14.596022    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h4jf5" podStartSLOduration=4.429738041 podStartE2EDuration="23.596002572s" totalImagesPullingTime="19.166264531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:14.596002745 +0000 UTC m=+24.782843405" watchObservedRunningTime="2026-07-27 21:55:14.596002572 +0000 UTC m=+24.782843233"
Jul 27 21:55:15.572107 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:15.572071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"b4d8155642084bc17b6d0df88030100492a6abce34b3b40b8bbc42d62a31ca3a"}
Jul 27 21:55:15.574342 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:15.574315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72" event={"ID":"aa83414d-a19d-4577-8fb4-8e209e89c167","Type":"ContainerStarted","Data":"a94de5fa7ae9382ccea82cc17c56cf720ddfeb76310daa3716432db256e80052"}
Jul 27 21:55:15.601932 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:15.599472    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-prm72" podStartSLOduration=3.685058982 podStartE2EDuration="24.59945175s" totalImagesPullingTime="20.914392768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:15.599340517 +0000 UTC m=+25.786181193" watchObservedRunningTime="2026-07-27 21:55:15.59945175 +0000 UTC m=+25.786292410"
Jul 27 21:55:16.316703 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:16.316653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:16.316703 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:16.316687    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:16.316941 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:16.316711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:16.316941 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:16.316790    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:16.316941 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:16.316874    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:16.317099 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:16.316964    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:17.584167 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:17.583935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" event={"ID":"8798a1c7-4dca-494d-9ff3-de38f9496c46","Type":"ContainerStarted","Data":"035d1421bb89b64bfa7b8afbd9a2ad048d5d42f8e736c679088773e6026db266"}
Jul 27 21:55:18.317325 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.317290    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:18.317325 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.317314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:18.317550 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.317389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:18.317550 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:18.317397    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:18.317550 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:18.317483    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:18.317658 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:18.317554    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:18.553006 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.552918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:55:18.553627 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.553608    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:55:18.587688 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.587639    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="36d2d4bee6d27e6cb0d3e245084bb9a72f344eb120a3fede97f42a6c5bde8a52" exitCode=0
Jul 27 21:55:18.588357 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.587705    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"36d2d4bee6d27e6cb0d3e245084bb9a72f344eb120a3fede97f42a6c5bde8a52"}
Jul 27 21:55:18.588357 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.587975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:55:18.588743 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.588637    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c2nbr"
Jul 27 21:55:18.661939 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:18.661894    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm" podStartSLOduration=8.44847336 podStartE2EDuration="27.661878029s" totalImagesPullingTime="19.213404669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:18.661101007 +0000 UTC m=+28.847941677" watchObservedRunningTime="2026-07-27 21:55:18.661878029 +0000 UTC m=+28.848718687"
Jul 27 21:55:19.559686 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.559463    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qlxss"]
Jul 27 21:55:19.559903 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.559849    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:19.559980 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:19.559934    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:19.560165 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.560140    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xm4r"]
Jul 27 21:55:19.560271 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.560257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:19.560363 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:19.560342    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:19.561739 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.561721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shbbf"]
Jul 27 21:55:19.561805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.561791    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:19.561874 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:19.561858    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:19.590839 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.590743    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:19.590839 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.590807    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:19.591306 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.590847    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:19.607466 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.607445    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:19.608110 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:19.608090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:20.594741 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:20.594704    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="c25d0163d920d638a7fab76b999932f95224906f411a85ed0843fcac20d77c1c" exitCode=0
Jul 27 21:55:20.595110 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:20.594759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"c25d0163d920d638a7fab76b999932f95224906f411a85ed0843fcac20d77c1c"}
Jul 27 21:55:21.316581 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:21.316550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:21.316766 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:21.316695    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:21.316766 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:21.316703    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:21.316766 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:21.316727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:21.316871 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:21.316812    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:21.316905 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:21.316877    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:22.602183 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:22.601954    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="f738d290f7490db73ac9cc41a443b50821830820a7283e3aa8113d90bdaad186" exitCode=0
Jul 27 21:55:22.602183 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:22.602019    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"f738d290f7490db73ac9cc41a443b50821830820a7283e3aa8113d90bdaad186"}
Jul 27 21:55:23.317148 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.317120    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:23.317148 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.317160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:23.317437 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.317231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:23.317437 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.317246    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:23.317437 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.317327    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:23.317571 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.317433    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:23.987968 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.987935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.987990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:23.988013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988110    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988199    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret podName:e6d161cf-b051-4a5c-98f0-ac23f474b98b nodeName:}" failed. No retries permitted until 2026-07-27 21:55:55.988168879 +0000 UTC m=+66.175009532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret") pod "global-pull-secret-syncer-qlxss" (UID: "e6d161cf-b051-4a5c-98f0-ac23f474b98b") : object "kube-system"/"original-pull-secret" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988110    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988270    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:55.988252658 +0000 UTC m=+66.175093297 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988116    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988300    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988313    2578 projected.go:196] Error preparing data for projected volume kube-api-access-9x9cw for pod openshift-network-diagnostics/network-check-target-shbbf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:55:23.988386 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:23.988355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw podName:2f88a5d8-a61a-4466-95c4-d37f3dad98bc nodeName:}" failed. No retries permitted until 2026-07-27 21:55:55.988343962 +0000 UTC m=+66.175184602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9x9cw" (UniqueName: "kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw") pod "network-check-target-shbbf" (UID: "2f88a5d8-a61a-4466-95c4-d37f3dad98bc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 21:55:25.316997 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.316959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:25.317569 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.317003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:25.317569 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.317003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:25.317569 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:25.317105    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xm4r" podUID="d8edead7-ce5c-4c47-80d1-bf5cb9d53525"
Jul 27 21:55:25.317569 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:25.317182    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qlxss" podUID="e6d161cf-b051-4a5c-98f0-ac23f474b98b"
Jul 27 21:55:25.317569 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:25.317250    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shbbf" podUID="2f88a5d8-a61a-4466-95c4-d37f3dad98bc"
Jul 27 21:55:25.645703 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.645606    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-91.ec2.internal" event="NodeReady"
Jul 27 21:55:25.645862 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.645765    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 21:55:25.697642 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.697605    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wzglt"]
Jul 27 21:55:25.701663 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.701634    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-967mv"]
Jul 27 21:55:25.701807 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.701790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.704823 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.704797    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 21:55:25.704952 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.704866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.704998 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.704954    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jz2cl\""
Jul 27 21:55:25.705034 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.705026    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 21:55:25.707591 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.707571    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 21:55:25.707723 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.707593    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-967mv"]
Jul 27 21:55:25.707841 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.707827    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 21:55:25.707903 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.707887    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4sqs\""
Jul 27 21:55:25.708064 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.708047    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 21:55:25.712997 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.712977    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wzglt"]
Jul 27 21:55:25.802543 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.802543 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f6lr\" (UniqueName: \"kubernetes.io/projected/652a8763-7cbf-4c08-abdb-0faabcaf7d28-kube-api-access-8f6lr\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.802788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/652a8763-7cbf-4c08-abdb-0faabcaf7d28-tmp-dir\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.802788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/652a8763-7cbf-4c08-abdb-0faabcaf7d28-config-volume\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.802788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e666f0d7-2854-46f3-b9db-748d7489971e-cert\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.802788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.802745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g89nh\" (UniqueName: \"kubernetes.io/projected/e666f0d7-2854-46f3-b9db-748d7489971e-kube-api-access-g89nh\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.903974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.903877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.903974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.903929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f6lr\" (UniqueName: \"kubernetes.io/projected/652a8763-7cbf-4c08-abdb-0faabcaf7d28-kube-api-access-8f6lr\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.903974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.903967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/652a8763-7cbf-4c08-abdb-0faabcaf7d28-tmp-dir\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.904243 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.903985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/652a8763-7cbf-4c08-abdb-0faabcaf7d28-config-volume\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.904243 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.903999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e666f0d7-2854-46f3-b9db-748d7489971e-cert\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.904243 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.904019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g89nh\" (UniqueName: \"kubernetes.io/projected/e666f0d7-2854-46f3-b9db-748d7489971e-kube-api-access-g89nh\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.904243 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:25.904023    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:25.904243 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:25.904095    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:26.404072605 +0000 UTC m=+36.590913255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:25.904576 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.904534    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/652a8763-7cbf-4c08-abdb-0faabcaf7d28-tmp-dir\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.909926 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.909905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e666f0d7-2854-46f3-b9db-748d7489971e-cert\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.914991 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.914956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/652a8763-7cbf-4c08-abdb-0faabcaf7d28-config-volume\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:25.915119 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.915070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g89nh\" (UniqueName: \"kubernetes.io/projected/e666f0d7-2854-46f3-b9db-748d7489971e-kube-api-access-g89nh\") pod \"ingress-canary-967mv\" (UID: \"e666f0d7-2854-46f3-b9db-748d7489971e\") " pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:25.916953 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:25.916931    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f6lr\" (UniqueName: \"kubernetes.io/projected/652a8763-7cbf-4c08-abdb-0faabcaf7d28-kube-api-access-8f6lr\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:26.021313 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:26.021281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-967mv"
Jul 27 21:55:26.407941 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:26.407906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:26.408603 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:26.408103    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:26.408603 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:26.408177    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:27.408160819 +0000 UTC m=+37.595001472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:26.962559 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:26.962368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-967mv"]
Jul 27 21:55:27.005113 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:27.005084    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode666f0d7_2854_46f3_b9db_748d7489971e.slice/crio-5616fab7bfa1d25f74228d2b0aa5ac3c4317d0112f899f7d1a62c55320e694bd WatchSource:0}: Error finding container 5616fab7bfa1d25f74228d2b0aa5ac3c4317d0112f899f7d1a62c55320e694bd: Status 404 returned error can't find the container with id 5616fab7bfa1d25f74228d2b0aa5ac3c4317d0112f899f7d1a62c55320e694bd
Jul 27 21:55:27.316885 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.316855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:27.317065 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.316974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:27.317065 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.317003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:27.320059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.320034    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 21:55:27.320173 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.320042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 21:55:27.321207 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.321188    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tm84h\""
Jul 27 21:55:27.321361 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.321235    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 21:55:27.321361 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.321293    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 21:55:27.321556 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.321541    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8ffd2\""
Jul 27 21:55:27.416524 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.416492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:27.416974 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:27.416634    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:27.416974 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:27.416734    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:29.416712394 +0000 UTC m=+39.603553034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:27.615723 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.615627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-967mv" event={"ID":"e666f0d7-2854-46f3-b9db-748d7489971e","Type":"ContainerStarted","Data":"5616fab7bfa1d25f74228d2b0aa5ac3c4317d0112f899f7d1a62c55320e694bd"}
Jul 27 21:55:27.619149 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.619118    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="c486b4994358317d44dc176b9d72f4be875b355afbec2c25b61b2da24e0a24bc" exitCode=0
Jul 27 21:55:27.619313 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:27.619189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"c486b4994358317d44dc176b9d72f4be875b355afbec2c25b61b2da24e0a24bc"}
Jul 27 21:55:28.628837 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:28.628803    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fe713f4-7663-4694-83df-feab1b0a9202" containerID="5f0c01aaba3fd5bf71df56e1031c00edd605f5e5cc86f484e454256fd73db659" exitCode=0
Jul 27 21:55:28.629351 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:28.628862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerDied","Data":"5f0c01aaba3fd5bf71df56e1031c00edd605f5e5cc86f484e454256fd73db659"}
Jul 27 21:55:29.436080 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:29.435837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:29.436080 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:29.435991    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:29.436335 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:29.436125    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:33.436104078 +0000 UTC m=+43.622944716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:29.633912 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:29.633876    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" event={"ID":"4fe713f4-7663-4694-83df-feab1b0a9202","Type":"ContainerStarted","Data":"f9186bc4d9aee862b821330ace0a38781c35acfb5da7a6cf17592731e8d99fce"}
Jul 27 21:55:29.635276 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:29.635247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-967mv" event={"ID":"e666f0d7-2854-46f3-b9db-748d7489971e","Type":"ContainerStarted","Data":"c44cb025c69a1aed8bc6c8e58e458205c821b967ea218d26c9fa4951e9fc1a33"}
Jul 27 21:55:29.658201 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:29.658147    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sc5mp" podStartSLOduration=8.704441158 podStartE2EDuration="38.658128258s" totalImagesPullingTime="29.9536871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:29.658041928 +0000 UTC m=+39.844882587" watchObservedRunningTime="2026-07-27 21:55:29.658128258 +0000 UTC m=+39.844968918"
Jul 27 21:55:29.676790 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:29.676752    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-967mv" podStartSLOduration=2.5744091669999998 podStartE2EDuration="4.676736449s" totalImagesPullingTime="2.102327282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:29.67613517 +0000 UTC m=+39.862975830" watchObservedRunningTime="2026-07-27 21:55:29.676736449 +0000 UTC m=+39.863577109"
Jul 27 21:55:33.465875 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:33.465833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:33.466239 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:33.465953    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:33.466239 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:33.466014    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:41.465996012 +0000 UTC m=+51.652836652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:36.129404 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.129370    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-5rpfw"]
Jul 27 21:55:36.134732 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.134711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.137507 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.137487    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 21:55:36.137608 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.137563    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 21:55:36.137780 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.137761    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 21:55:36.140691 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.140652    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h8nll\""
Jul 27 21:55:36.140792 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.140758    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 21:55:36.143313 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.143296    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 21:55:36.144956 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.144936    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-5rpfw"]
Jul 27 21:55:36.285662 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eda4dc46-cd71-49b2-a7d1-1915a6a86610-serving-cert\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.285803 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-service-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.285803 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285754    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-tmp\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.285890 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpwwm\" (UniqueName: \"kubernetes.io/projected/eda4dc46-cd71-49b2-a7d1-1915a6a86610-kube-api-access-fpwwm\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.285890 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-snapshots\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.285890 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.285860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eda4dc46-cd71-49b2-a7d1-1915a6a86610-serving-cert\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-service-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-tmp\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpwwm\" (UniqueName: \"kubernetes.io/projected/eda4dc46-cd71-49b2-a7d1-1915a6a86610-kube-api-access-fpwwm\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-snapshots\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.387959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.389774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.389358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.390581 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.390559    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-tmp\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.390760 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.390741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/eda4dc46-cd71-49b2-a7d1-1915a6a86610-snapshots\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.390813 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.390763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eda4dc46-cd71-49b2-a7d1-1915a6a86610-service-ca-bundle\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.394113 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.394083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eda4dc46-cd71-49b2-a7d1-1915a6a86610-serving-cert\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.404358 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.404338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpwwm\" (UniqueName: \"kubernetes.io/projected/eda4dc46-cd71-49b2-a7d1-1915a6a86610-kube-api-access-fpwwm\") pod \"insights-operator-5885fdb759-5rpfw\" (UID: \"eda4dc46-cd71-49b2-a7d1-1915a6a86610\") " pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.444178 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.444160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-5rpfw"
Jul 27 21:55:36.586186 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.586155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-5rpfw"]
Jul 27 21:55:36.589105 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:36.589075    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda4dc46_cd71_49b2_a7d1_1915a6a86610.slice/crio-c33fcbeb6483cec69559bffe623a76cb2887898bf902f3ee140d4e53b9f8f86a WatchSource:0}: Error finding container c33fcbeb6483cec69559bffe623a76cb2887898bf902f3ee140d4e53b9f8f86a: Status 404 returned error can't find the container with id c33fcbeb6483cec69559bffe623a76cb2887898bf902f3ee140d4e53b9f8f86a
Jul 27 21:55:36.653835 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:36.653766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-5rpfw" event={"ID":"eda4dc46-cd71-49b2-a7d1-1915a6a86610","Type":"ContainerStarted","Data":"c33fcbeb6483cec69559bffe623a76cb2887898bf902f3ee140d4e53b9f8f86a"}
Jul 27 21:55:39.662310 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:39.662276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-5rpfw" event={"ID":"eda4dc46-cd71-49b2-a7d1-1915a6a86610","Type":"ContainerStarted","Data":"99882c3bde3e3fd101deb3088b511bb0e41b031897c6c4e0742f0a5cb5869e10"}
Jul 27 21:55:40.511275 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.511212    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-5rpfw" podStartSLOduration=2.119864483 podStartE2EDuration="4.511187614s" totalImagesPullingTime="2.391323131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:39.683308887 +0000 UTC m=+49.870149547" watchObservedRunningTime="2026-07-27 21:55:40.511187614 +0000 UTC m=+50.698028275"
Jul 27 21:55:40.511746 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.511730    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kpnlq"]
Jul 27 21:55:40.525731 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.525706    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.531184 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.530164    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kpnlq"]
Jul 27 21:55:40.531184 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.530594    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5pg8\""
Jul 27 21:55:40.531184 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.530919    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 21:55:40.531184 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.531138    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 21:55:40.719456 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.719421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e7ad512-7204-4cbc-9c3d-a8054d155126-crio-socket\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.719904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.719478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e7ad512-7204-4cbc-9c3d-a8054d155126-data-volume\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.719904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.719495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.719904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.719516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.719904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.719538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9fb\" (UniqueName: \"kubernetes.io/projected/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-api-access-gv9fb\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820468    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9fb\" (UniqueName: \"kubernetes.io/projected/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-api-access-gv9fb\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820706 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:40.820524    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:55:40.820706 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:40.820562    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:40.820706 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:40.820596    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:41.320576672 +0000 UTC m=+51.507417328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : configmap "kube-rbac-proxy" not found
Jul 27 21:55:40.820706 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:40.820628    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:41.320608842 +0000 UTC m=+51.507449482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:40.820706 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e7ad512-7204-4cbc-9c3d-a8054d155126-crio-socket\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820938 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e7ad512-7204-4cbc-9c3d-a8054d155126-data-volume\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.820938 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.820922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e7ad512-7204-4cbc-9c3d-a8054d155126-crio-socket\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.821088 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.821071    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e7ad512-7204-4cbc-9c3d-a8054d155126-data-volume\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:40.831930 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:40.831907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9fb\" (UniqueName: \"kubernetes.io/projected/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-api-access-gv9fb\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:41.324460 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:41.324426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:41.324618 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:41.324469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:41.324618 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.324555    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:55:41.324717 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.324631    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:42.324615171 +0000 UTC m=+52.511455808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : configmap "kube-rbac-proxy" not found
Jul 27 21:55:41.324717 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.324648    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:41.324793 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.324717    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:42.324700818 +0000 UTC m=+52.511541455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:41.526512 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:41.526477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:41.526657 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.526631    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:41.526716 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:41.526706    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:57.526691578 +0000 UTC m=+67.713532215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:55:42.334271 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:42.334244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:42.334741 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:42.334327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:42.334741 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:42.334386    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:55:42.334741 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:42.334423    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:42.334741 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:42.334437    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:44.334423624 +0000 UTC m=+54.521264262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : configmap "kube-rbac-proxy" not found
Jul 27 21:55:42.334741 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:42.334500    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:44.33448004 +0000 UTC m=+54.521320678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:42.662617 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:42.662542    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pxm6b_9f091a7e-c114-4d02-968a-9cd9f8f404eb/dns-node-resolver/0.log"
Jul 27 21:55:43.063190 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:43.063160    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6v88q_75184c67-8fdc-4da3-9aa8-97dcb9f948d8/node-ca/0.log"
Jul 27 21:55:43.862662 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:43.862638    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-967mv_e666f0d7-2854-46f3-b9db-748d7489971e/serve-healthcheck-canary/0.log"
Jul 27 21:55:44.352700 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:44.352652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:44.352862 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:44.352707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:44.352862 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:44.352775    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:55:44.352862 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:44.352831    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:48.352818103 +0000 UTC m=+58.539658739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : configmap "kube-rbac-proxy" not found
Jul 27 21:55:44.352973 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:44.352890    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:44.352973 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:44.352958    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:48.352944384 +0000 UTC m=+58.539785021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:46.017633 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.017597    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"]
Jul 27 21:55:46.047261 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.047238    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"]
Jul 27 21:55:46.047394 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.047332    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.049881 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.049861    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 21:55:46.049881 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.049871    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-456vb\""
Jul 27 21:55:46.050923 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.050907    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 21:55:46.051019 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.050929    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:55:46.065124 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.065102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ftpb\" (UniqueName: \"kubernetes.io/projected/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-kube-api-access-6ftpb\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.065213 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.065155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.121011 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.120978    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"]
Jul 27 21:55:46.135719 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.135694    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"]
Jul 27 21:55:46.135856 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.135784    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.138808 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.138787    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:55:46.138897 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.138832    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fcj4h\""
Jul 27 21:55:46.139090 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.139075    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 21:55:46.139148 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.139131    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 21:55:46.139183 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.139131    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 21:55:46.145885 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.145864    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zmwg9"]
Jul 27 21:55:46.146010 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.145995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.148798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.148779    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:55:46.148895 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.148820    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 21:55:46.148895 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.148782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-k4khj\""
Jul 27 21:55:46.149083 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.149063    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 21:55:46.149191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.149091    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 21:55:46.160372 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.160354    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"]
Jul 27 21:55:46.160460 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.160382    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"]
Jul 27 21:55:46.160460 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.160423    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.161271 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.161250    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zmwg9"]
Jul 27 21:55:46.163919 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.163893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qqzfv\""
Jul 27 21:55:46.164016 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.163947    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 21:55:46.164016 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.163898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 21:55:46.164169 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.164094    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:55:46.164169 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.164102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 21:55:46.165626 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ftpb\" (UniqueName: \"kubernetes.io/projected/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-kube-api-access-6ftpb\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.165626 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.165626 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-config\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z94rl\" (UniqueName: \"kubernetes.io/projected/8db60bac-31d7-481a-9f6a-53789dc1be2f-kube-api-access-z94rl\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc378c26-a97b-4322-bd34-823a4b84ed49-config\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc378c26-a97b-4322-bd34-823a4b84ed49-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2j5\" (UniqueName: \"kubernetes.io/projected/bc378c26-a97b-4322-bd34-823a4b84ed49-kube-api-access-kl2j5\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.165904 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-274hh\" (UniqueName: \"kubernetes.io/projected/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-kube-api-access-274hh\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.166309 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.165995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-trusted-ca\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.166309 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.166046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8db60bac-31d7-481a-9f6a-53789dc1be2f-serving-cert\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.166309 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:46.166181    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:55:46.166309 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:46.166245    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls podName:75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:46.666225489 +0000 UTC m=+56.853066128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls") pod "cluster-samples-operator-6748489656-pfqjv" (UID: "75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6") : secret "samples-operator-tls" not found
Jul 27 21:55:46.169855 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.169838    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 21:55:46.175495 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.175472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ftpb\" (UniqueName: \"kubernetes.io/projected/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-kube-api-access-6ftpb\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.266972 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.266944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2j5\" (UniqueName: \"kubernetes.io/projected/bc378c26-a97b-4322-bd34-823a4b84ed49-kube-api-access-kl2j5\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.267095 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.266976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.267095 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.266991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-274hh\" (UniqueName: \"kubernetes.io/projected/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-kube-api-access-274hh\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.267095 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-trusted-ca\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.267236 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8db60bac-31d7-481a-9f6a-53789dc1be2f-serving-cert\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.267236 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.267236 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-config\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.267379 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z94rl\" (UniqueName: \"kubernetes.io/projected/8db60bac-31d7-481a-9f6a-53789dc1be2f-kube-api-access-z94rl\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.267379 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc378c26-a97b-4322-bd34-823a4b84ed49-config\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.267535 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc378c26-a97b-4322-bd34-823a4b84ed49-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.267898 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.267962 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc378c26-a97b-4322-bd34-823a4b84ed49-config\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.267998 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.267982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-config\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.268346 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.268283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8db60bac-31d7-481a-9f6a-53789dc1be2f-trusted-ca\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.269586 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.269560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.269759 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.269741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8db60bac-31d7-481a-9f6a-53789dc1be2f-serving-cert\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.269895 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.269877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc378c26-a97b-4322-bd34-823a4b84ed49-serving-cert\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.276028 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.275995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2j5\" (UniqueName: \"kubernetes.io/projected/bc378c26-a97b-4322-bd34-823a4b84ed49-kube-api-access-kl2j5\") pod \"service-ca-operator-5d6c75f84f-7p7bq\" (UID: \"bc378c26-a97b-4322-bd34-823a4b84ed49\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.276166 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.276146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-274hh\" (UniqueName: \"kubernetes.io/projected/3d7130e9-1bcb-4489-86eb-2abfe7f9479a-kube-api-access-274hh\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7\" (UID: \"3d7130e9-1bcb-4489-86eb-2abfe7f9479a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.276337 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.276322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z94rl\" (UniqueName: \"kubernetes.io/projected/8db60bac-31d7-481a-9f6a-53789dc1be2f-kube-api-access-z94rl\") pod \"console-operator-584dcb6b54-zmwg9\" (UID: \"8db60bac-31d7-481a-9f6a-53789dc1be2f\") " pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.445686 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.445648    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"
Jul 27 21:55:46.454813 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.454787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"
Jul 27 21:55:46.468915 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.468892    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:46.603300 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.603268    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7"]
Jul 27 21:55:46.609264 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:46.609234    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d7130e9_1bcb_4489_86eb_2abfe7f9479a.slice/crio-f2eeae7f2cb0088ed5479f1ee956c10ffb3e1452a025cd8d6ff76c7bc3c3f17c WatchSource:0}: Error finding container f2eeae7f2cb0088ed5479f1ee956c10ffb3e1452a025cd8d6ff76c7bc3c3f17c: Status 404 returned error can't find the container with id f2eeae7f2cb0088ed5479f1ee956c10ffb3e1452a025cd8d6ff76c7bc3c3f17c
Jul 27 21:55:46.623170 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.623150    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq"]
Jul 27 21:55:46.625507 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:46.625477    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc378c26_a97b_4322_bd34_823a4b84ed49.slice/crio-5d9779f5f927341c4414200f43e00731d4650a5d0267984feae37a9e0da78d3d WatchSource:0}: Error finding container 5d9779f5f927341c4414200f43e00731d4650a5d0267984feae37a9e0da78d3d: Status 404 returned error can't find the container with id 5d9779f5f927341c4414200f43e00731d4650a5d0267984feae37a9e0da78d3d
Jul 27 21:55:46.647299 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.647270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-zmwg9"]
Jul 27 21:55:46.650473 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:46.650446    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db60bac_31d7_481a_9f6a_53789dc1be2f.slice/crio-7cb811cf00e307aa6feb1465e70eb511667ced10e3848904eda526124642d73d WatchSource:0}: Error finding container 7cb811cf00e307aa6feb1465e70eb511667ced10e3848904eda526124642d73d: Status 404 returned error can't find the container with id 7cb811cf00e307aa6feb1465e70eb511667ced10e3848904eda526124642d73d
Jul 27 21:55:46.671260 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.671239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:46.671419 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:46.671401    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:55:46.671475 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:46.671465    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls podName:75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:47.671444589 +0000 UTC m=+57.858285241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls") pod "cluster-samples-operator-6748489656-pfqjv" (UID: "75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6") : secret "samples-operator-tls" not found
Jul 27 21:55:46.681042 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.681017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" event={"ID":"8db60bac-31d7-481a-9f6a-53789dc1be2f","Type":"ContainerStarted","Data":"7cb811cf00e307aa6feb1465e70eb511667ced10e3848904eda526124642d73d"}
Jul 27 21:55:46.682167 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.682147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq" event={"ID":"bc378c26-a97b-4322-bd34-823a4b84ed49","Type":"ContainerStarted","Data":"5d9779f5f927341c4414200f43e00731d4650a5d0267984feae37a9e0da78d3d"}
Jul 27 21:55:46.683203 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:46.683183    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7" event={"ID":"3d7130e9-1bcb-4489-86eb-2abfe7f9479a","Type":"ContainerStarted","Data":"f2eeae7f2cb0088ed5479f1ee956c10ffb3e1452a025cd8d6ff76c7bc3c3f17c"}
Jul 27 21:55:47.679544 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:47.678953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:47.679544 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:47.679151    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:55:47.679544 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:47.679228    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls podName:75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:49.679208787 +0000 UTC m=+59.866049435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls") pod "cluster-samples-operator-6748489656-pfqjv" (UID: "75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6") : secret "samples-operator-tls" not found
Jul 27 21:55:48.384200 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:48.384163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:48.384371 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:48.384294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:48.384371 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:48.384321    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:48.384371 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:48.384370    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:55:48.384529 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:48.384385    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:56.384366173 +0000 UTC m=+66.571206811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:48.384529 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:48.384420    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:56.384405059 +0000 UTC m=+66.571245719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : configmap "kube-rbac-proxy" not found
Jul 27 21:55:49.693713 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:49.693679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:49.694185 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:49.693779    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:55:49.694185 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:49.693854    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls podName:75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6 nodeName:}" failed. No retries permitted until 2026-07-27 21:55:53.693828318 +0000 UTC m=+63.880668957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls") pod "cluster-samples-operator-6748489656-pfqjv" (UID: "75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6") : secret "samples-operator-tls" not found
Jul 27 21:55:51.612606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.612571    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wvjqm"
Jul 27 21:55:51.696954 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.696914    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7" event={"ID":"3d7130e9-1bcb-4489-86eb-2abfe7f9479a","Type":"ContainerStarted","Data":"9ae4d73e100346a788daba6d7c69700f2b01e28beea8abbcd39bd9d045d1c4ae"}
Jul 27 21:55:51.698505 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.698486    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/0.log"
Jul 27 21:55:51.698614 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.698523    2578 generic.go:356] "Generic (PLEG): container finished" podID="8db60bac-31d7-481a-9f6a-53789dc1be2f" containerID="be9b53377d6b937ff7af7c9a5679c3a46c310512431bb2c02e6afb30cdfdb4c0" exitCode=255
Jul 27 21:55:51.698614 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.698553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" event={"ID":"8db60bac-31d7-481a-9f6a-53789dc1be2f","Type":"ContainerDied","Data":"be9b53377d6b937ff7af7c9a5679c3a46c310512431bb2c02e6afb30cdfdb4c0"}
Jul 27 21:55:51.698777 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.698762    2578 scope.go:117] "RemoveContainer" containerID="be9b53377d6b937ff7af7c9a5679c3a46c310512431bb2c02e6afb30cdfdb4c0"
Jul 27 21:55:51.700138 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.700120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq" event={"ID":"bc378c26-a97b-4322-bd34-823a4b84ed49","Type":"ContainerStarted","Data":"c93ca5a6aa04ca5270cdffa677d046c78dc64385787a26c87b0046be7b591760"}
Jul 27 21:55:51.717405 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.717361    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7" podStartSLOduration=1.736703256 podStartE2EDuration="5.717344168s" totalImagesPullingTime="3.980640912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:51.715959734 +0000 UTC m=+61.902800392" watchObservedRunningTime="2026-07-27 21:55:51.717344168 +0000 UTC m=+61.904184829"
Jul 27 21:55:51.748846 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:51.748806    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq" podStartSLOduration=1.784509777 podStartE2EDuration="5.748791825s" totalImagesPullingTime="3.964282048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:51.748241431 +0000 UTC m=+61.935082092" watchObservedRunningTime="2026-07-27 21:55:51.748791825 +0000 UTC m=+61.935632484"
Jul 27 21:55:52.705044 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705014    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/1.log"
Jul 27 21:55:52.705471 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705451    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/0.log"
Jul 27 21:55:52.705513 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705487    2578 generic.go:356] "Generic (PLEG): container finished" podID="8db60bac-31d7-481a-9f6a-53789dc1be2f" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239" exitCode=255
Jul 27 21:55:52.705589 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705567    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" event={"ID":"8db60bac-31d7-481a-9f6a-53789dc1be2f","Type":"ContainerDied","Data":"ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"}
Jul 27 21:55:52.705653 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705627    2578 scope.go:117] "RemoveContainer" containerID="be9b53377d6b937ff7af7c9a5679c3a46c310512431bb2c02e6afb30cdfdb4c0"
Jul 27 21:55:52.705866 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:52.705836    2578 scope.go:117] "RemoveContainer" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"
Jul 27 21:55:52.706032 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:52.706014    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:55:53.709495 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:53.709467    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/1.log"
Jul 27 21:55:53.709907 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:53.709803    2578 scope.go:117] "RemoveContainer" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"
Jul 27 21:55:53.709969 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:53.709950    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:55:53.725477 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:53.725447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:55:53.725615 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:53.725570    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 21:55:53.725663 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:53.725648    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls podName:75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6 nodeName:}" failed. No retries permitted until 2026-07-27 21:56:01.725626056 +0000 UTC m=+71.912466695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls") pod "cluster-samples-operator-6748489656-pfqjv" (UID: "75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6") : secret "samples-operator-tls" not found
Jul 27 21:55:56.042928 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.042894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:55:56.043345 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.042973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:56.043345 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.043000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:56.045957 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.045940    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 21:55:56.046043 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.046011    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 21:55:56.046093 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.046017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 21:55:56.054202 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:56.054182    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 21:55:56.054289 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:56.054262    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs podName:d8edead7-ce5c-4c47-80d1-bf5cb9d53525 nodeName:}" failed. No retries permitted until 2026-07-27 21:57:00.054246234 +0000 UTC m=+130.241086873 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs") pod "network-metrics-daemon-4xm4r" (UID: "d8edead7-ce5c-4c47-80d1-bf5cb9d53525") : secret "metrics-daemon-secret" not found
Jul 27 21:55:56.056564 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.056535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6d161cf-b051-4a5c-98f0-ac23f474b98b-original-pull-secret\") pod \"global-pull-secret-syncer-qlxss\" (UID: \"e6d161cf-b051-4a5c-98f0-ac23f474b98b\") " pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:56.056645 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.056622    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 21:55:56.067272 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.067248    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x9cw\" (UniqueName: \"kubernetes.io/projected/2f88a5d8-a61a-4466-95c4-d37f3dad98bc-kube-api-access-9x9cw\") pod \"network-check-target-shbbf\" (UID: \"2f88a5d8-a61a-4466-95c4-d37f3dad98bc\") " pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:56.126512 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.126465    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qlxss"
Jul 27 21:55:56.135481 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.135461    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8ffd2\""
Jul 27 21:55:56.142992 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.142970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:55:56.265341 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.265303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qlxss"]
Jul 27 21:55:56.268054 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:56.268035    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d161cf_b051_4a5c_98f0_ac23f474b98b.slice/crio-35b61199bbde404c55cccc8a3e667e05dae370ac2b7d74d658892929b964126b WatchSource:0}: Error finding container 35b61199bbde404c55cccc8a3e667e05dae370ac2b7d74d658892929b964126b: Status 404 returned error can't find the container with id 35b61199bbde404c55cccc8a3e667e05dae370ac2b7d74d658892929b964126b
Jul 27 21:55:56.281746 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.281724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shbbf"]
Jul 27 21:55:56.284571 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:55:56.284548    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f88a5d8_a61a_4466_95c4_d37f3dad98bc.slice/crio-17bf12abf18d310d0b33d184e687f70bf63f4a0b1474a47df56ebb12277b48d9 WatchSource:0}: Error finding container 17bf12abf18d310d0b33d184e687f70bf63f4a0b1474a47df56ebb12277b48d9: Status 404 returned error can't find the container with id 17bf12abf18d310d0b33d184e687f70bf63f4a0b1474a47df56ebb12277b48d9
Jul 27 21:55:56.445109 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.445035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:56.445109 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.445074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:56.445266 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:56.445168    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:56.445266 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:56.445218    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls podName:7e7ad512-7204-4cbc-9c3d-a8054d155126 nodeName:}" failed. No retries permitted until 2026-07-27 21:56:12.445204161 +0000 UTC m=+82.632044799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpnlq" (UID: "7e7ad512-7204-4cbc-9c3d-a8054d155126") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:55:56.445505 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.445485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e7ad512-7204-4cbc-9c3d-a8054d155126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:55:56.469889 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.469867    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:56.469989 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.469922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:55:56.470204 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.470193    2578 scope.go:117] "RemoveContainer" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"
Jul 27 21:55:56.470345 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:56.470329    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:55:56.719434 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.719352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shbbf" event={"ID":"2f88a5d8-a61a-4466-95c4-d37f3dad98bc","Type":"ContainerStarted","Data":"17bf12abf18d310d0b33d184e687f70bf63f4a0b1474a47df56ebb12277b48d9"}
Jul 27 21:55:56.721494 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:56.721467    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qlxss" event={"ID":"e6d161cf-b051-4a5c-98f0-ac23f474b98b","Type":"ContainerStarted","Data":"35b61199bbde404c55cccc8a3e667e05dae370ac2b7d74d658892929b964126b"}
Jul 27 21:55:57.554309 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:55:57.554269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:55:57.554723 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:57.554450    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:55:57.554723 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:55:57.554525    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls podName:652a8763-7cbf-4c08-abdb-0faabcaf7d28 nodeName:}" failed. No retries permitted until 2026-07-27 21:56:29.554503588 +0000 UTC m=+99.741344258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls") pod "dns-default-wzglt" (UID: "652a8763-7cbf-4c08-abdb-0faabcaf7d28") : secret "dns-default-metrics-tls" not found
Jul 27 21:56:00.735358 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:00.735293    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shbbf" event={"ID":"2f88a5d8-a61a-4466-95c4-d37f3dad98bc","Type":"ContainerStarted","Data":"095e9e37bc286f72c2e4c8544a6b51918eed8cfb7b10c4edda245b891cfa45dc"}
Jul 27 21:56:00.735824 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:00.735403    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:56:00.736937 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:00.736895    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qlxss" event={"ID":"e6d161cf-b051-4a5c-98f0-ac23f474b98b","Type":"ContainerStarted","Data":"d6dd4accef67be10b8dae2bbbefabeb9d022b27b82e273176ec3500d49eaa8cb"}
Jul 27 21:56:00.770539 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:00.770494    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-shbbf" podStartSLOduration=65.4427143 podStartE2EDuration="1m9.770480906s" totalImagesPullingTime="4.327766606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:00.751240988 +0000 UTC m=+70.938081649" watchObservedRunningTime="2026-07-27 21:56:00.770480906 +0000 UTC m=+70.957321564"
Jul 27 21:56:00.771221 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:00.771184    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qlxss" podStartSLOduration=65.423502558 podStartE2EDuration="1m9.771169106s" totalImagesPullingTime="4.347666548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:00.770243234 +0000 UTC m=+70.957083893" watchObservedRunningTime="2026-07-27 21:56:00.771169106 +0000 UTC m=+70.958009766"
Jul 27 21:56:01.788821 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:01.788789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:56:01.791176 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:01.791156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-pfqjv\" (UID: \"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:56:01.958856 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:01.958827    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-456vb\""
Jul 27 21:56:01.966347 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:01.966328    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"
Jul 27 21:56:02.090684 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:02.090642    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv"]
Jul 27 21:56:02.746281 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:02.746240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv" event={"ID":"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6","Type":"ContainerStarted","Data":"1fbff017df53c256b58bb05bafaf3eacd1e41631f272991aaff03f832d55b4f2"}
Jul 27 21:56:04.754075 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:04.754042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv" event={"ID":"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6","Type":"ContainerStarted","Data":"852e36e0b39f4b54f13bfad4a29ddc236f56c6a06733d5291e856dca2b67ea0b"}
Jul 27 21:56:04.754490 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:04.754090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv" event={"ID":"75fe8dab-4e71-4ff4-93a7-6f3f4501e8c6","Type":"ContainerStarted","Data":"91e8c146e4f8d7820b63198d9103097011b93d46099e1e059038bbb818bdcb6f"}
Jul 27 21:56:04.771044 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:04.771001    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-pfqjv" podStartSLOduration=17.15965529 podStartE2EDuration="18.770986361s" totalImagesPullingTime="1.611331071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:04.770058094 +0000 UTC m=+74.956898752" watchObservedRunningTime="2026-07-27 21:56:04.770986361 +0000 UTC m=+74.957827054"
Jul 27 21:56:08.317366 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.317336    2578 scope.go:117] "RemoveContainer" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"
Jul 27 21:56:08.767294 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.767265    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 21:56:08.767711 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.767695    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/1.log"
Jul 27 21:56:08.767786 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.767734    2578 generic.go:356] "Generic (PLEG): container finished" podID="8db60bac-31d7-481a-9f6a-53789dc1be2f" containerID="a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976" exitCode=255
Jul 27 21:56:08.767828 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.767781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" event={"ID":"8db60bac-31d7-481a-9f6a-53789dc1be2f","Type":"ContainerDied","Data":"a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976"}
Jul 27 21:56:08.767873 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.767863    2578 scope.go:117] "RemoveContainer" containerID="ecbfd089a43a7b64bd6bfc318ae17dd4062dc30ec744356e9fa1fc04057f5239"
Jul 27 21:56:08.768226 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:08.768202    2578 scope.go:117] "RemoveContainer" containerID="a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976"
Jul 27 21:56:08.768426 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:08.768405    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:56:09.772558 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:09.772531    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 21:56:12.472053 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.471911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:56:12.474321 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.474301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e7ad512-7204-4cbc-9c3d-a8054d155126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpnlq\" (UID: \"7e7ad512-7204-4cbc-9c3d-a8054d155126\") " pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:56:12.642080 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.642046    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5pg8\""
Jul 27 21:56:12.649500 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.649481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kpnlq"
Jul 27 21:56:12.771828 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.771636    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kpnlq"]
Jul 27 21:56:12.774522 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:12.774496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7ad512_7204_4cbc_9c3d_a8054d155126.slice/crio-4bcb4b076393505e04713ef76df761befe178688039373dab092425d2af86d36 WatchSource:0}: Error finding container 4bcb4b076393505e04713ef76df761befe178688039373dab092425d2af86d36: Status 404 returned error can't find the container with id 4bcb4b076393505e04713ef76df761befe178688039373dab092425d2af86d36
Jul 27 21:56:12.786360 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:12.786329    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpnlq" event={"ID":"7e7ad512-7204-4cbc-9c3d-a8054d155126","Type":"ContainerStarted","Data":"4bcb4b076393505e04713ef76df761befe178688039373dab092425d2af86d36"}
Jul 27 21:56:13.688079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.686956    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-96cb4d677-xv2nw"]
Jul 27 21:56:13.690637 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.690613    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"]
Jul 27 21:56:13.690798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.690760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.694605 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.694151    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.698825 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.698806    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 21:56:13.699064 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.699043    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 21:56:13.699774 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.699744    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 21:56:13.699926 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.699912    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-65kmq\""
Jul 27 21:56:13.700162 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.700147    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 21:56:13.700237 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.700218    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 21:56:13.700447 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.700432    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 21:56:13.701048 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.701027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 21:56:13.701048 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.701041    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mt8ct\""
Jul 27 21:56:13.710181 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.710161    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 21:56:13.713284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.713235    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"]
Jul 27 21:56:13.729993 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.729963    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-96cb4d677-xv2nw"]
Jul 27 21:56:13.782620 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.782579    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56g4q\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.782845 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.782799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.782911 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.782844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.783030 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rsz\" (UniqueName: \"kubernetes.io/projected/7a58a6d7-f2da-4718-b64f-096961d2d3b1-kube-api-access-v4rsz\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.783082 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a58a6d7-f2da-4718-b64f-096961d2d3b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.783220 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.783285 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.783340 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.783408 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.783467 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.783422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.845562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.845528    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96cb4d677-xv2nw"]
Jul 27 21:56:13.845813 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:13.845789    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-56g4q registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-96cb4d677-xv2nw" podUID="4e818468-c1cf-448f-8ab4-3451c380e5fc"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56g4q\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884549    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rsz\" (UniqueName: \"kubernetes.io/projected/7a58a6d7-f2da-4718-b64f-096961d2d3b1-kube-api-access-v4rsz\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a58a6d7-f2da-4718-b64f-096961d2d3b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884755    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.884988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.884858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.885578 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.885311    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.886492 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.886467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.887526 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.887480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.888000 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.887977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.888475 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.888451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a58a6d7-f2da-4718-b64f-096961d2d3b1-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.889646 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.889626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.889777 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.889756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.893607 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.893588    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54cc4f975-ccxw2"]
Jul 27 21:56:13.898213 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.898194    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.905036 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.905007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.905883 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.905860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56g4q\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q\") pod \"image-registry-96cb4d677-xv2nw\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") " pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:13.906047 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.906028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rsz\" (UniqueName: \"kubernetes.io/projected/7a58a6d7-f2da-4718-b64f-096961d2d3b1-kube-api-access-v4rsz\") pod \"managed-serviceaccount-addon-agent-8559c874f8-trqsr\" (UID: \"7a58a6d7-f2da-4718-b64f-096961d2d3b1\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:13.909219 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.909196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54cc4f975-ccxw2"]
Jul 27 21:56:13.986038 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llqmj\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-kube-api-access-llqmj\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986240 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-tls\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986240 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-trusted-ca\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986240 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-image-registry-private-configuration\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986240 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/950c2c66-ead4-460e-a1d1-be6d1193e09b-ca-trust-extracted\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986406 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-installation-pull-secrets\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986406 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-bound-sa-token\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:13.986406 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:13.986353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-certificates\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.019546 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.019518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"
Jul 27 21:56:14.087659 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-installation-pull-secrets\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.087861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-bound-sa-token\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.087861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-certificates\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.087861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llqmj\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-kube-api-access-llqmj\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.087861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-tls\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.087861 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-trusted-ca\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.088147 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-image-registry-private-configuration\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.088147 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.087927    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/950c2c66-ead4-460e-a1d1-be6d1193e09b-ca-trust-extracted\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.088550 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.088525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/950c2c66-ead4-460e-a1d1-be6d1193e09b-ca-trust-extracted\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.089332 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.089302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-trusted-ca\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.089514 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.089487    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-certificates\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.091431 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.091400    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-image-registry-private-configuration\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.091934 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.091911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/950c2c66-ead4-460e-a1d1-be6d1193e09b-installation-pull-secrets\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.092161 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.092136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-registry-tls\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.097553 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.097528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llqmj\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-kube-api-access-llqmj\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.098331 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.098295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/950c2c66-ead4-460e-a1d1-be6d1193e09b-bound-sa-token\") pod \"image-registry-54cc4f975-ccxw2\" (UID: \"950c2c66-ead4-460e-a1d1-be6d1193e09b\") " pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.174967 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.174938    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr"]
Jul 27 21:56:14.178430 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:14.178400    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a58a6d7_f2da_4718_b64f_096961d2d3b1.slice/crio-cd26d9f33e91068a6ac8aab179eeee9e145fa56780c21151c3ed4ad0f6b44e3f WatchSource:0}: Error finding container cd26d9f33e91068a6ac8aab179eeee9e145fa56780c21151c3ed4ad0f6b44e3f: Status 404 returned error can't find the container with id cd26d9f33e91068a6ac8aab179eeee9e145fa56780c21151c3ed4ad0f6b44e3f
Jul 27 21:56:14.221240 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.221209    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.366988 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.366960    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54cc4f975-ccxw2"]
Jul 27 21:56:14.371100 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:14.371074    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950c2c66_ead4_460e_a1d1_be6d1193e09b.slice/crio-106ca2e78cd3323220995705812793db8e187884201de4a3da310fad3fcc4e06 WatchSource:0}: Error finding container 106ca2e78cd3323220995705812793db8e187884201de4a3da310fad3fcc4e06: Status 404 returned error can't find the container with id 106ca2e78cd3323220995705812793db8e187884201de4a3da310fad3fcc4e06
Jul 27 21:56:14.795887 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.795825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr" event={"ID":"7a58a6d7-f2da-4718-b64f-096961d2d3b1","Type":"ContainerStarted","Data":"cd26d9f33e91068a6ac8aab179eeee9e145fa56780c21151c3ed4ad0f6b44e3f"}
Jul 27 21:56:14.797638 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.797617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:14.797788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.797635    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" event={"ID":"950c2c66-ead4-460e-a1d1-be6d1193e09b","Type":"ContainerStarted","Data":"3f72995ba82d529f7168bda01f90efdb057abf7153bfb72a9b4b439b42209219"}
Jul 27 21:56:14.797788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.797747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" event={"ID":"950c2c66-ead4-460e-a1d1-be6d1193e09b","Type":"ContainerStarted","Data":"106ca2e78cd3323220995705812793db8e187884201de4a3da310fad3fcc4e06"}
Jul 27 21:56:14.797788 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.797778    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:14.805248 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.805214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:14.837951 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.837898    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" podStartSLOduration=1.837878582 podStartE2EDuration="1.837878582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:14.836379999 +0000 UTC m=+85.023220658" watchObservedRunningTime="2026-07-27 21:56:14.837878582 +0000 UTC m=+85.024719240"
Jul 27 21:56:14.899937 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.899898    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900101 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.899979    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900101 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900023    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56g4q\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900101 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900267 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900104    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900267 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900130    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900267 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900167    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900267 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900209    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration\") pod \"4e818468-c1cf-448f-8ab4-3451c380e5fc\" (UID: \"4e818468-c1cf-448f-8ab4-3451c380e5fc\") "
Jul 27 21:56:14.900557 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900526    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:56:14.901459 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.900899    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:56:14.901738 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.901654    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:56:14.903996 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.903954    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:56:14.904264 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.904211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:56:14.904511 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.904428    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:56:14.904511 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.904485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q" (OuterVolumeSpecName: "kube-api-access-56g4q") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "kube-api-access-56g4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:56:14.913747 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:14.913576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4e818468-c1cf-448f-8ab4-3451c380e5fc" (UID: "4e818468-c1cf-448f-8ab4-3451c380e5fc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001413    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-image-registry-private-configuration\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001450    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-tls\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001468    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-registry-certificates\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001483    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56g4q\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-kube-api-access-56g4q\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001496    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e818468-c1cf-448f-8ab4-3451c380e5fc-bound-sa-token\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001509    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e818468-c1cf-448f-8ab4-3451c380e5fc-installation-pull-secrets\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001522    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e818468-c1cf-448f-8ab4-3451c380e5fc-ca-trust-extracted\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.001562 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.001536    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e818468-c1cf-448f-8ab4-3451c380e5fc-trusted-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:56:15.801063 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.801032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-96cb4d677-xv2nw"
Jul 27 21:56:15.834375 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.834352    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-96cb4d677-xv2nw"]
Jul 27 21:56:15.837963 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:15.837928    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-96cb4d677-xv2nw"]
Jul 27 21:56:16.322402 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:16.322369    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e818468-c1cf-448f-8ab4-3451c380e5fc" path="/var/lib/kubelet/pods/4e818468-c1cf-448f-8ab4-3451c380e5fc/volumes"
Jul 27 21:56:16.469741 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:16.469708    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:56:16.469922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:16.469775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:56:16.470205 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:16.470180    2578 scope.go:117] "RemoveContainer" containerID="a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976"
Jul 27 21:56:16.470402 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:16.470382    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:56:18.752960 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.752926    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r75c7"]
Jul 27 21:56:18.756171 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.756150    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.760391 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760369    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 27 21:56:18.760513 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760398    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 21:56:18.760513 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760437    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 21:56:18.760513 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760369    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 21:56:18.760513 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760370    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 27 21:56:18.760718 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.760651    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7457v\""
Jul 27 21:56:18.766338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.766317    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r75c7"]
Jul 27 21:56:18.831053 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.831023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.831201 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.831102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cps5t\" (UniqueName: \"kubernetes.io/projected/2c42892f-7c7b-410e-8890-c95e64da3d8d-kube-api-access-cps5t\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.831201 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.831128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.831201 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.831150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c42892f-7c7b-410e-8890-c95e64da3d8d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.932732 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.932489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cps5t\" (UniqueName: \"kubernetes.io/projected/2c42892f-7c7b-410e-8890-c95e64da3d8d-kube-api-access-cps5t\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.932732 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.932542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.932732 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.932572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c42892f-7c7b-410e-8890-c95e64da3d8d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.932732 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.932610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.933477 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.933429    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c42892f-7c7b-410e-8890-c95e64da3d8d-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.935544 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.935515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.935623 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.935528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c42892f-7c7b-410e-8890-c95e64da3d8d-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:18.942548 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:18.942527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cps5t\" (UniqueName: \"kubernetes.io/projected/2c42892f-7c7b-410e-8890-c95e64da3d8d-kube-api-access-cps5t\") pod \"prometheus-operator-84d69f794d-r75c7\" (UID: \"2c42892f-7c7b-410e-8890-c95e64da3d8d\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:19.080438 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.080404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7"
Jul 27 21:56:19.220879 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.220856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r75c7"]
Jul 27 21:56:19.223356 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:19.223330    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c42892f_7c7b_410e_8890_c95e64da3d8d.slice/crio-c6e1eae2ec6d75824d1a2ba3b83ab07f27017ce3917630936f1661d855669746 WatchSource:0}: Error finding container c6e1eae2ec6d75824d1a2ba3b83ab07f27017ce3917630936f1661d855669746: Status 404 returned error can't find the container with id c6e1eae2ec6d75824d1a2ba3b83ab07f27017ce3917630936f1661d855669746
Jul 27 21:56:19.818130 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.818049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpnlq" event={"ID":"7e7ad512-7204-4cbc-9c3d-a8054d155126","Type":"ContainerStarted","Data":"e126d0b592387aafe20382a7c2cb8429bbbe6ab0650090f909c59df551ffdfbd"}
Jul 27 21:56:19.819572 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.819544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7" event={"ID":"2c42892f-7c7b-410e-8890-c95e64da3d8d","Type":"ContainerStarted","Data":"c6e1eae2ec6d75824d1a2ba3b83ab07f27017ce3917630936f1661d855669746"}
Jul 27 21:56:19.821627 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.821101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr" event={"ID":"7a58a6d7-f2da-4718-b64f-096961d2d3b1","Type":"ContainerStarted","Data":"cc3659a92873d8723530452d5d07d5746639eecead93725b1b6d5af2f51d2e78"}
Jul 27 21:56:19.837492 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:19.837449    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8559c874f8-trqsr" podStartSLOduration=1.997656018 podStartE2EDuration="6.837431419s" totalImagesPullingTime="4.839775401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:19.836770674 +0000 UTC m=+90.023611335" watchObservedRunningTime="2026-07-27 21:56:19.837431419 +0000 UTC m=+90.024272079"
Jul 27 21:56:22.836533 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:22.836503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7" event={"ID":"2c42892f-7c7b-410e-8890-c95e64da3d8d","Type":"ContainerStarted","Data":"ca76ee141ebea8d7a7959123ee88477543f041e8536a970ab1483d355dddc841"}
Jul 27 21:56:23.843270 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:23.843231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpnlq" event={"ID":"7e7ad512-7204-4cbc-9c3d-a8054d155126","Type":"ContainerStarted","Data":"7384d9ef79ac2276d46d8a12e05f97f430f6b886c108809cd2484a54e9a96c69"}
Jul 27 21:56:23.845191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:23.845162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7" event={"ID":"2c42892f-7c7b-410e-8890-c95e64da3d8d","Type":"ContainerStarted","Data":"c9427a197a658cd019e36bbc3d568a000ee9652c0c14fb8e629b56dbb9c6fdff"}
Jul 27 21:56:23.862922 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:23.862867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-r75c7" podStartSLOduration=2.332075139 podStartE2EDuration="5.862850098s" totalImagesPullingTime="3.530774959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:23.862133737 +0000 UTC m=+94.048974431" watchObservedRunningTime="2026-07-27 21:56:23.862850098 +0000 UTC m=+94.049690758"
Jul 27 21:56:26.858855 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:26.858821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpnlq" event={"ID":"7e7ad512-7204-4cbc-9c3d-a8054d155126","Type":"ContainerStarted","Data":"16288deef428240d9ed54a9342eef7e9d724dfc80119c547f9b3516ba23a22cc"}
Jul 27 21:56:26.877070 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:26.877025    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kpnlq" podStartSLOduration=33.411896272999996 podStartE2EDuration="46.877009585s" totalImagesPullingTime="13.465113312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:55:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:26.875863595 +0000 UTC m=+97.062704276" watchObservedRunningTime="2026-07-27 21:56:26.877009585 +0000 UTC m=+97.063850246"
Jul 27 21:56:28.317188 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:28.317159    2578 scope.go:117] "RemoveContainer" containerID="a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976"
Jul 27 21:56:28.317524 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:28.317310    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-zmwg9_openshift-console-operator(8db60bac-31d7-481a-9f6a-53789dc1be2f)\"" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podUID="8db60bac-31d7-481a-9f6a-53789dc1be2f"
Jul 27 21:56:29.626095 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:29.626052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:56:29.628272 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:29.628249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/652a8763-7cbf-4c08-abdb-0faabcaf7d28-metrics-tls\") pod \"dns-default-wzglt\" (UID: \"652a8763-7cbf-4c08-abdb-0faabcaf7d28\") " pod="openshift-dns/dns-default-wzglt"
Jul 27 21:56:29.916783 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:29.916698    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jz2cl\""
Jul 27 21:56:29.924913 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:29.924883    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wzglt"
Jul 27 21:56:30.050178 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.050157    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wzglt"]
Jul 27 21:56:30.052327 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:30.052304    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652a8763_7cbf_4c08_abdb_0faabcaf7d28.slice/crio-6260f8ec37726388a570cf6eac10cea80047fc22399ae2590d7db992a809152f WatchSource:0}: Error finding container 6260f8ec37726388a570cf6eac10cea80047fc22399ae2590d7db992a809152f: Status 404 returned error can't find the container with id 6260f8ec37726388a570cf6eac10cea80047fc22399ae2590d7db992a809152f
Jul 27 21:56:30.188091 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.188010    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-68z7n"]
Jul 27 21:56:30.192521 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.192499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.195027 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.195007    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 21:56:30.195126 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.195048    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jqbc5\""
Jul 27 21:56:30.195322 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.195296    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 21:56:30.195545 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.195530    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 21:56:30.330931 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.330903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-sys\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.330931 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.330931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-root\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.330950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.330971    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-wtmp\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.331033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4k72\" (UniqueName: \"kubernetes.io/projected/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-kube-api-access-s4k72\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.331064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.331088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-textfile\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.331109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-tls\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.331125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.331123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-metrics-client-ca\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431610 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-tls\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-metrics-client-ca\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-sys\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-root\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.431798 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-wtmp\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-sys\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-root\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4k72\" (UniqueName: \"kubernetes.io/projected/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-kube-api-access-s4k72\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-wtmp\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432059 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.431973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-textfile\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.432281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-textfile\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.432299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-metrics-client-ca\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.432338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.432299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.434108 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.434087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-tls\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.434108 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.434106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.440155 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.440105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4k72\" (UniqueName: \"kubernetes.io/projected/4f6294bc-568c-4ed9-9a61-7be11bbb1e7d-kube-api-access-s4k72\") pod \"node-exporter-68z7n\" (UID: \"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d\") " pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.501814 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.501786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68z7n"
Jul 27 21:56:30.512802 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:30.512773    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6294bc_568c_4ed9_9a61_7be11bbb1e7d.slice/crio-c6bafd514f8428a4637c93fecfd1625334bf9d13457ed789c7613a487a0c0548 WatchSource:0}: Error finding container c6bafd514f8428a4637c93fecfd1625334bf9d13457ed789c7613a487a0c0548: Status 404 returned error can't find the container with id c6bafd514f8428a4637c93fecfd1625334bf9d13457ed789c7613a487a0c0548
Jul 27 21:56:30.873436 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.873398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68z7n" event={"ID":"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d","Type":"ContainerStarted","Data":"c6bafd514f8428a4637c93fecfd1625334bf9d13457ed789c7613a487a0c0548"}
Jul 27 21:56:30.877006 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:30.875421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzglt" event={"ID":"652a8763-7cbf-4c08-abdb-0faabcaf7d28","Type":"ContainerStarted","Data":"6260f8ec37726388a570cf6eac10cea80047fc22399ae2590d7db992a809152f"}
Jul 27 21:56:31.744579 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:31.744549    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-shbbf"
Jul 27 21:56:32.883790 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.883701    2578 generic.go:356] "Generic (PLEG): container finished" podID="4f6294bc-568c-4ed9-9a61-7be11bbb1e7d" containerID="87d4348529ee1576215ee4a843e46419962d929c032fefcf313cd25b79958e25" exitCode=0
Jul 27 21:56:32.884242 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.883784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68z7n" event={"ID":"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d","Type":"ContainerDied","Data":"87d4348529ee1576215ee4a843e46419962d929c032fefcf313cd25b79958e25"}
Jul 27 21:56:32.885534 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.885511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzglt" event={"ID":"652a8763-7cbf-4c08-abdb-0faabcaf7d28","Type":"ContainerStarted","Data":"e070ece482abe888bf0476a8b07525316c00c799de0ce660d90b89703f5b8858"}
Jul 27 21:56:32.885623 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.885551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wzglt" event={"ID":"652a8763-7cbf-4c08-abdb-0faabcaf7d28","Type":"ContainerStarted","Data":"ce224d4798f43b708b21d9bd67cbd006bba9aa2061781753e4f01266b26771d7"}
Jul 27 21:56:32.885623 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.885573    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wzglt"
Jul 27 21:56:32.925851 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:32.925798    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wzglt" podStartSLOduration=65.685934065 podStartE2EDuration="1m7.925779133s" totalImagesPullingTime="2.239845068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:32.925087526 +0000 UTC m=+103.111928183" watchObservedRunningTime="2026-07-27 21:56:32.925779133 +0000 UTC m=+103.112619793"
Jul 27 21:56:33.891291 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:33.891252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68z7n" event={"ID":"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d","Type":"ContainerStarted","Data":"c7b395b12728a2c048cc8fb4681909de4e607b862fced25f230d18ef56423438"}
Jul 27 21:56:33.891778 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:33.891320    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68z7n" event={"ID":"4f6294bc-568c-4ed9-9a61-7be11bbb1e7d","Type":"ContainerStarted","Data":"a66d6ed4dfe4bb1bff7256482c92423380a58962342766d3950c2a28d015d68c"}
Jul 27 21:56:33.911738 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:33.911696    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-68z7n" podStartSLOduration=2.030338257 podStartE2EDuration="3.911659518s" totalImagesPullingTime="1.781328415s" totalInitContainerRuntime="99.992846ms" isStatefulPod=true podCreationTimestamp="2026-07-27 21:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:33.91020883 +0000 UTC m=+104.097049489" watchObservedRunningTime="2026-07-27 21:56:33.911659518 +0000 UTC m=+104.098500177"
Jul 27 21:56:34.225744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.225639    2578 patch_prober.go:28] interesting pod/image-registry-54cc4f975-ccxw2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:56:34.225744 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.225724    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" podUID="950c2c66-ead4-460e-a1d1-be6d1193e09b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:56:34.951862 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.951826    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"]
Jul 27 21:56:34.978916 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.978890    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"]
Jul 27 21:56:34.979057 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.978980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:34.981347 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.981324    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 27 21:56:34.981451 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:34.981368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8xkfs\""
Jul 27 21:56:35.072828 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.072790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.072973 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.072876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x922h\" (UniqueName: \"kubernetes.io/projected/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-kube-api-access-x922h\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.174495 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.174457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.174693 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.174541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x922h\" (UniqueName: \"kubernetes.io/projected/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-kube-api-access-x922h\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.174693 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:35.174613    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 27 21:56:35.174767 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:56:35.174708    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert podName:e76c5bfe-5735-4b0a-8c4b-2070e2f0587a nodeName:}" failed. No retries permitted until 2026-07-27 21:56:35.674662877 +0000 UTC m=+105.861503514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-rls2f" (UID: "e76c5bfe-5735-4b0a-8c4b-2070e2f0587a") : secret "monitoring-plugin-cert" not found
Jul 27 21:56:35.184600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.184575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x922h\" (UniqueName: \"kubernetes.io/projected/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-kube-api-access-x922h\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.378665 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.378631    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"]
Jul 27 21:56:35.396094 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.396061    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"]
Jul 27 21:56:35.396244 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.396192    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.399013 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.398866    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 27 21:56:35.399013 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.398878    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 27 21:56:35.399013 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.398866    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2mwl6\""
Jul 27 21:56:35.399013 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.398957    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 27 21:56:35.399300 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.399235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 27 21:56:35.399360 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.399311    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 27 21:56:35.403105 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.403076    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 27 21:56:35.577992 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.577957    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578159 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578159 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-metrics-client-ca\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578159 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578159 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jxs\" (UniqueName: \"kubernetes.io/projected/c54c377e-825b-4421-a9ba-3f500c424040-kube-api-access-k6jxs\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-serving-certs-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.578284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.578236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-federate-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679158 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jxs\" (UniqueName: \"kubernetes.io/projected/c54c377e-825b-4421-a9ba-3f500c424040-kube-api-access-k6jxs\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679158 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-serving-certs-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679381 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679381 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-federate-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679381 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679561 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679561 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-metrics-client-ca\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679561 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.679561 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.679924 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.679901    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-serving-certs-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.680361 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.680331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.680656 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.680629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c377e-825b-4421-a9ba-3f500c424040-metrics-client-ca\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.682140 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.682114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-telemeter-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.682225 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.682110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.682466 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.682445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-federate-client-tls\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.682505 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.682493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e76c5bfe-5735-4b0a-8c4b-2070e2f0587a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-rls2f\" (UID: \"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.682545 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.682493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c54c377e-825b-4421-a9ba-3f500c424040-secret-telemeter-client\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.688095 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.688074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jxs\" (UniqueName: \"kubernetes.io/projected/c54c377e-825b-4421-a9ba-3f500c424040-kube-api-access-k6jxs\") pod \"telemeter-client-cf5ff4b67-bwt52\" (UID: \"c54c377e-825b-4421-a9ba-3f500c424040\") " pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.706991 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.706968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"
Jul 27 21:56:35.807466 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.807431    2578 patch_prober.go:28] interesting pod/image-registry-54cc4f975-ccxw2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:56:35.807730 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.807488    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" podUID="950c2c66-ead4-460e-a1d1-be6d1193e09b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:56:35.838876 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.838849    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cf5ff4b67-bwt52"]
Jul 27 21:56:35.845188 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:35.845159    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54c377e_825b_4421_a9ba_3f500c424040.slice/crio-f021fbbec1c9d7925d52a23a2df6591afc8832ae225c1332246977a0b73555f9 WatchSource:0}: Error finding container f021fbbec1c9d7925d52a23a2df6591afc8832ae225c1332246977a0b73555f9: Status 404 returned error can't find the container with id f021fbbec1c9d7925d52a23a2df6591afc8832ae225c1332246977a0b73555f9
Jul 27 21:56:35.888436 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.888406    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:35.898610 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:35.898578    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52" event={"ID":"c54c377e-825b-4421-a9ba-3f500c424040","Type":"ContainerStarted","Data":"f021fbbec1c9d7925d52a23a2df6591afc8832ae225c1332246977a0b73555f9"}
Jul 27 21:56:36.010722 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:36.010700    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"]
Jul 27 21:56:36.013062 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:36.013033    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode76c5bfe_5735_4b0a_8c4b_2070e2f0587a.slice/crio-68d3c2cc7fc32892d88579e5bd1968f1324fd4e95185e0fb6be201d16569a045 WatchSource:0}: Error finding container 68d3c2cc7fc32892d88579e5bd1968f1324fd4e95185e0fb6be201d16569a045: Status 404 returned error can't find the container with id 68d3c2cc7fc32892d88579e5bd1968f1324fd4e95185e0fb6be201d16569a045
Jul 27 21:56:36.904462 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:36.904416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f" event={"ID":"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a","Type":"ContainerStarted","Data":"68d3c2cc7fc32892d88579e5bd1968f1324fd4e95185e0fb6be201d16569a045"}
Jul 27 21:56:38.917723 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:38.917658    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52" event={"ID":"c54c377e-825b-4421-a9ba-3f500c424040","Type":"ContainerStarted","Data":"dc38bf04c041df1940141f4157da0175e88cd0a4b77eb6e2a312659131badd83"}
Jul 27 21:56:38.919906 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:38.919871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f" event={"ID":"e76c5bfe-5735-4b0a-8c4b-2070e2f0587a","Type":"ContainerStarted","Data":"baa8606dc46718f23108934f93c19f8128c31c9b05145fb732541914b26ea0a7"}
Jul 27 21:56:38.920149 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:38.920123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:38.926088 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:38.926063    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f"
Jul 27 21:56:38.935448 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:38.935406    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-rls2f" podStartSLOduration=3.553102969 podStartE2EDuration="4.935394591s" totalImagesPullingTime="1.382291622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:38.93454254 +0000 UTC m=+109.121383199" watchObservedRunningTime="2026-07-27 21:56:38.935394591 +0000 UTC m=+109.122235250"
Jul 27 21:56:39.925626 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:39.925587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52" event={"ID":"c54c377e-825b-4421-a9ba-3f500c424040","Type":"ContainerStarted","Data":"70fdc557e406e53b27bd6010b7d8459771208c2dbb05e74544da52c626a87bb3"}
Jul 27 21:56:39.926125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:39.925657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52" event={"ID":"c54c377e-825b-4421-a9ba-3f500c424040","Type":"ContainerStarted","Data":"fae19477fab1dc1e95da458e83416d8f3ca72f6a72d0a40be42f7530e26a524a"}
Jul 27 21:56:39.948261 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:39.948203    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cf5ff4b67-bwt52" podStartSLOduration=1.8673291490000001 podStartE2EDuration="4.948183981s" totalImagesPullingTime="3.080854832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:39.945957349 +0000 UTC m=+110.132798009" watchObservedRunningTime="2026-07-27 21:56:39.948183981 +0000 UTC m=+110.135024640"
Jul 27 21:56:42.893848 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:42.893811    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wzglt"
Jul 27 21:56:43.316844 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.316815    2578 scope.go:117] "RemoveContainer" containerID="a21d831bb53b5cf14cd087696e9b2edf0e9a262d611046b292096bd02b410976"
Jul 27 21:56:43.944366 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.944340    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 21:56:43.944874 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.944417    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" event={"ID":"8db60bac-31d7-481a-9f6a-53789dc1be2f","Type":"ContainerStarted","Data":"fb062ac349c8b4bc9dc71a0cc70328ab4fa88e4daf1592be1428d1bb601e3e4f"}
Jul 27 21:56:43.944874 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.944698    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:56:43.959713 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.959661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9"
Jul 27 21:56:43.965315 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:43.965266    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-zmwg9" podStartSLOduration=54.020699954 podStartE2EDuration="57.965245647s" totalImagesPullingTime="3.944545693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:43.964123705 +0000 UTC m=+114.150964376" watchObservedRunningTime="2026-07-27 21:56:43.965245647 +0000 UTC m=+114.152086306"
Jul 27 21:56:44.139250 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.139219    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-nnmzd"]
Jul 27 21:56:44.142287 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.142270    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:56:44.145825 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.145803    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 27 21:56:44.145924 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.145847    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 27 21:56:44.145924 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.145809    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5gggx\""
Jul 27 21:56:44.164468 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.164445    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-nnmzd"]
Jul 27 21:56:44.225403 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.225336    2578 patch_prober.go:28] interesting pod/image-registry-54cc4f975-ccxw2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 27 21:56:44.225510 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.225408    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2" podUID="950c2c66-ead4-460e-a1d1-be6d1193e09b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 21:56:44.249949 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.249923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x877\" (UniqueName: \"kubernetes.io/projected/0393ec85-e28b-4f36-b3e6-3ecf99f8c619-kube-api-access-2x877\") pod \"downloads-7c97dc4555-nnmzd\" (UID: \"0393ec85-e28b-4f36-b3e6-3ecf99f8c619\") " pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:56:44.350301 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.350279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x877\" (UniqueName: \"kubernetes.io/projected/0393ec85-e28b-4f36-b3e6-3ecf99f8c619-kube-api-access-2x877\") pod \"downloads-7c97dc4555-nnmzd\" (UID: \"0393ec85-e28b-4f36-b3e6-3ecf99f8c619\") " pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:56:44.362165 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.362139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x877\" (UniqueName: \"kubernetes.io/projected/0393ec85-e28b-4f36-b3e6-3ecf99f8c619-kube-api-access-2x877\") pod \"downloads-7c97dc4555-nnmzd\" (UID: \"0393ec85-e28b-4f36-b3e6-3ecf99f8c619\") " pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:56:44.451126 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.451092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:56:44.582183 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.582155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-nnmzd"]
Jul 27 21:56:44.585227 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:56:44.585202    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0393ec85_e28b_4f36_b3e6_3ecf99f8c619.slice/crio-e87a35c1d6ce54f64656f8d4686fc8c7b74b05f3ed4f0b525f03308b0a648fe3 WatchSource:0}: Error finding container e87a35c1d6ce54f64656f8d4686fc8c7b74b05f3ed4f0b525f03308b0a648fe3: Status 404 returned error can't find the container with id e87a35c1d6ce54f64656f8d4686fc8c7b74b05f3ed4f0b525f03308b0a648fe3
Jul 27 21:56:44.950545 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.950506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-nnmzd" event={"ID":"0393ec85-e28b-4f36-b3e6-3ecf99f8c619","Type":"ContainerStarted","Data":"e87a35c1d6ce54f64656f8d4686fc8c7b74b05f3ed4f0b525f03308b0a648fe3"}
Jul 27 21:56:44.952262 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.952230    2578 generic.go:356] "Generic (PLEG): container finished" podID="eda4dc46-cd71-49b2-a7d1-1915a6a86610" containerID="99882c3bde3e3fd101deb3088b511bb0e41b031897c6c4e0742f0a5cb5869e10" exitCode=0
Jul 27 21:56:44.952408 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.952323    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-5rpfw" event={"ID":"eda4dc46-cd71-49b2-a7d1-1915a6a86610","Type":"ContainerDied","Data":"99882c3bde3e3fd101deb3088b511bb0e41b031897c6c4e0742f0a5cb5869e10"}
Jul 27 21:56:44.953089 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:44.952817    2578 scope.go:117] "RemoveContainer" containerID="99882c3bde3e3fd101deb3088b511bb0e41b031897c6c4e0742f0a5cb5869e10"
Jul 27 21:56:45.807505 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:45.807478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54cc4f975-ccxw2"
Jul 27 21:56:45.961180 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:45.960564    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-5rpfw" event={"ID":"eda4dc46-cd71-49b2-a7d1-1915a6a86610","Type":"ContainerStarted","Data":"0873176c19a042185ec4856698d092d8bd06d642593913dc78b55f6040cce08f"}
Jul 27 21:56:47.364985 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:47.364935    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzglt_652a8763-7cbf-4c08-abdb-0faabcaf7d28/dns/0.log"
Jul 27 21:56:47.565128 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:47.565096    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzglt_652a8763-7cbf-4c08-abdb-0faabcaf7d28/kube-rbac-proxy/0.log"
Jul 27 21:56:48.164273 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:48.164158    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pxm6b_9f091a7e-c114-4d02-968a-9cd9f8f404eb/dns-node-resolver/0.log"
Jul 27 21:56:48.365504 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:48.365475    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54cc4f975-ccxw2_950c2c66-ead4-460e-a1d1-be6d1193e09b/registry/0.log"
Jul 27 21:56:48.764961 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:48.764934    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6v88q_75184c67-8fdc-4da3-9aa8-97dcb9f948d8/node-ca/0.log"
Jul 27 21:56:49.564358 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:49.564318    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-967mv_e666f0d7-2854-46f3-b9db-748d7489971e/serve-healthcheck-canary/0.log"
Jul 27 21:56:54.750411 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.750373    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:56:54.755127 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.755100    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.758206 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.758179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 27 21:56:54.758330 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.758212    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 27 21:56:54.759233 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.759214    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-76gwq\""
Jul 27 21:56:54.759359 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.759338    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 27 21:56:54.759434 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.759379    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 27 21:56:54.759480 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.759469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 27 21:56:54.766115 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.766094    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:56:54.846522 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.846710 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.846710 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.846710 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf5q7\" (UniqueName: \"kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.846710 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.846911 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.846728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948163 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948341 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948341 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf5q7\" (UniqueName: \"kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948341 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948503 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.948503 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.948476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.949129 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.949106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.949252 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.949130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.949252 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.949197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.951284 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.951258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.952471 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.952449    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:54.957803 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:54.957784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf5q7\" (UniqueName: \"kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7\") pod \"console-5584bcbc7-54964\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") " pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:56:55.067020 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:56:55.066937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:57:00.100040 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:00.100009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:57:00.102661 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:00.102630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8edead7-ce5c-4c47-80d1-bf5cb9d53525-metrics-certs\") pod \"network-metrics-daemon-4xm4r\" (UID: \"d8edead7-ce5c-4c47-80d1-bf5cb9d53525\") " pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:57:00.339340 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:00.339312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tm84h\""
Jul 27 21:57:00.346313 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:00.346293    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xm4r"
Jul 27 21:57:01.024306 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:01.023014    2578 generic.go:356] "Generic (PLEG): container finished" podID="3d7130e9-1bcb-4489-86eb-2abfe7f9479a" containerID="9ae4d73e100346a788daba6d7c69700f2b01e28beea8abbcd39bd9d045d1c4ae" exitCode=0
Jul 27 21:57:01.024306 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:01.023059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7" event={"ID":"3d7130e9-1bcb-4489-86eb-2abfe7f9479a","Type":"ContainerDied","Data":"9ae4d73e100346a788daba6d7c69700f2b01e28beea8abbcd39bd9d045d1c4ae"}
Jul 27 21:57:01.024306 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:01.023435    2578 scope.go:117] "RemoveContainer" containerID="9ae4d73e100346a788daba6d7c69700f2b01e28beea8abbcd39bd9d045d1c4ae"
Jul 27 21:57:01.143876 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:01.143470    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xm4r"]
Jul 27 21:57:01.148448 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:57:01.148403    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8edead7_ce5c_4c47_80d1_bf5cb9d53525.slice/crio-86d01dd82854223cfb250ebfdf719a6dd12bd11430197d55bb0cb92d6fab3370 WatchSource:0}: Error finding container 86d01dd82854223cfb250ebfdf719a6dd12bd11430197d55bb0cb92d6fab3370: Status 404 returned error can't find the container with id 86d01dd82854223cfb250ebfdf719a6dd12bd11430197d55bb0cb92d6fab3370
Jul 27 21:57:01.169920 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:01.169893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:57:01.172698 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:57:01.172653    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6daea69_8c76_4fba_8960_d0908e4e3b96.slice/crio-d4fadbd9517677c5d26af5a9db9564cc2d4fb692aa2651eda241025f94e8664a WatchSource:0}: Error finding container d4fadbd9517677c5d26af5a9db9564cc2d4fb692aa2651eda241025f94e8664a: Status 404 returned error can't find the container with id d4fadbd9517677c5d26af5a9db9564cc2d4fb692aa2651eda241025f94e8664a
Jul 27 21:57:02.033218 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.032627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-nnmzd" event={"ID":"0393ec85-e28b-4f36-b3e6-3ecf99f8c619","Type":"ContainerStarted","Data":"1a6e036b624240a5b603c3384963b548b3a49f54ee9056fe4518ef75c3ae88ac"}
Jul 27 21:57:02.033218 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.033167    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:57:02.035242 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.035206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5584bcbc7-54964" event={"ID":"c6daea69-8c76-4fba-8960-d0908e4e3b96","Type":"ContainerStarted","Data":"d4fadbd9517677c5d26af5a9db9564cc2d4fb692aa2651eda241025f94e8664a"}
Jul 27 21:57:02.037486 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.037461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xm4r" event={"ID":"d8edead7-ce5c-4c47-80d1-bf5cb9d53525","Type":"ContainerStarted","Data":"86d01dd82854223cfb250ebfdf719a6dd12bd11430197d55bb0cb92d6fab3370"}
Jul 27 21:57:02.040517 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.040278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7" event={"ID":"3d7130e9-1bcb-4489-86eb-2abfe7f9479a","Type":"ContainerStarted","Data":"cf31982058a995088d6aa823d8556b9cfbde911436f018504ce3c3a60554200b"}
Jul 27 21:57:02.054374 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.054224    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-nnmzd" podStartSLOduration=1.557463305 podStartE2EDuration="18.054205579s" totalImagesPullingTime="16.496742274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:57:02.051769563 +0000 UTC m=+132.238610225" watchObservedRunningTime="2026-07-27 21:57:02.054205579 +0000 UTC m=+132.241046241"
Jul 27 21:57:02.054700 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:02.054622    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-nnmzd"
Jul 27 21:57:03.049195 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.049129    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xm4r" event={"ID":"d8edead7-ce5c-4c47-80d1-bf5cb9d53525","Type":"ContainerStarted","Data":"5d2b48f51749d6b964fc8e3e229a592320a5b00d43c4ca9aedf02db868b4010b"}
Jul 27 21:57:03.049195 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.049188    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xm4r" event={"ID":"d8edead7-ce5c-4c47-80d1-bf5cb9d53525","Type":"ContainerStarted","Data":"862960a5da177a4d97a4f31a28f4d8ac37b37bd9168373abde3329b5c07e3394"}
Jul 27 21:57:03.244583 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.244474    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4xm4r" podStartSLOduration=131.067798615 podStartE2EDuration="2m12.244422317s" totalImagesPullingTime="1.176623702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:57:03.069363394 +0000 UTC m=+133.256204050" watchObservedRunningTime="2026-07-27 21:57:03.244422317 +0000 UTC m=+133.431262977"
Jul 27 21:57:03.246433 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.246400    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:57:03.266515 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.266488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:57:03.266753 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.266636    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.276439 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.276247    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 27 21:57:03.332764 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.332580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.332764 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.332636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.333079 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.333051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhvhx\" (UniqueName: \"kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.333217 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.333100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.333217 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.333140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.333217 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.333166    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.333388 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.333228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434461 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhvhx\" (UniqueName: \"kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434639 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434639 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434780 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434780 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434885 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.434885 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.434814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.435302 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.435275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.435514 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.435484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.435948 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.435855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.435948 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.435854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.437979 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.437953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.437979 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.437971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.447592 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.447570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhvhx\" (UniqueName: \"kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx\") pod \"console-649bd87974-2c5dj\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") " pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:03.582289 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:03.582053    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:04.973024 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:04.972860    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:57:04.975662 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:57:04.975633    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b6de28_d954_4c72_9449_467fabfaa6f9.slice/crio-20a72d38536c1146a29c07d4a6e4691ef86d382535d5b4c53344bb24a7433c0a WatchSource:0}: Error finding container 20a72d38536c1146a29c07d4a6e4691ef86d382535d5b4c53344bb24a7433c0a: Status 404 returned error can't find the container with id 20a72d38536c1146a29c07d4a6e4691ef86d382535d5b4c53344bb24a7433c0a
Jul 27 21:57:05.060906 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:05.060864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649bd87974-2c5dj" event={"ID":"a6b6de28-d954-4c72-9449-467fabfaa6f9","Type":"ContainerStarted","Data":"20a72d38536c1146a29c07d4a6e4691ef86d382535d5b4c53344bb24a7433c0a"}
Jul 27 21:57:06.067446 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.067408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5584bcbc7-54964" event={"ID":"c6daea69-8c76-4fba-8960-d0908e4e3b96","Type":"ContainerStarted","Data":"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"}
Jul 27 21:57:06.069396 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.069361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649bd87974-2c5dj" event={"ID":"a6b6de28-d954-4c72-9449-467fabfaa6f9","Type":"ContainerStarted","Data":"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"}
Jul 27 21:57:06.071170 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.071139    2578 generic.go:356] "Generic (PLEG): container finished" podID="bc378c26-a97b-4322-bd34-823a4b84ed49" containerID="c93ca5a6aa04ca5270cdffa677d046c78dc64385787a26c87b0046be7b591760" exitCode=0
Jul 27 21:57:06.071303 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.071171    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq" event={"ID":"bc378c26-a97b-4322-bd34-823a4b84ed49","Type":"ContainerDied","Data":"c93ca5a6aa04ca5270cdffa677d046c78dc64385787a26c87b0046be7b591760"}
Jul 27 21:57:06.071530 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.071513    2578 scope.go:117] "RemoveContainer" containerID="c93ca5a6aa04ca5270cdffa677d046c78dc64385787a26c87b0046be7b591760"
Jul 27 21:57:06.105459 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.105404    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5584bcbc7-54964" podStartSLOduration=8.061059653 podStartE2EDuration="12.105380051s" totalImagesPullingTime="4.044320398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:57:06.087286563 +0000 UTC m=+136.274127280" watchObservedRunningTime="2026-07-27 21:57:06.105380051 +0000 UTC m=+136.292220711"
Jul 27 21:57:06.123655 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:06.123596    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-649bd87974-2c5dj" podStartSLOduration=2.444935209 podStartE2EDuration="3.123573183s" totalImagesPullingTime="678.637974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:57:06.120979879 +0000 UTC m=+136.307820554" watchObservedRunningTime="2026-07-27 21:57:06.123573183 +0000 UTC m=+136.310413844"
Jul 27 21:57:07.077963 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:07.077922    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-7p7bq" event={"ID":"bc378c26-a97b-4322-bd34-823a4b84ed49","Type":"ContainerStarted","Data":"a6c4d9f38d891dfdcbc72adfc78da9a8f5cd36208fdd12936ca73f0e3f9904b7"}
Jul 27 21:57:13.582911 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:13.582878    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:13.583413 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:13.582950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:13.587684 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:13.587646    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:14.110536 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:14.110509    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:57:14.160097 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:14.160070    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:57:15.067532 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:15.067495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:57:39.184480 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.184419    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5584bcbc7-54964" podUID="c6daea69-8c76-4fba-8960-d0908e4e3b96" containerName="console" containerID="cri-o://c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa" gracePeriod=15
Jul 27 21:57:39.441061 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.441002    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:57:39.539790 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf5q7\" (UniqueName: \"kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.539950 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.539950 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539857    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.539950 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539883    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.539950 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539916    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.539950 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.539939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert\") pod \"c6daea69-8c76-4fba-8960-d0908e4e3b96\" (UID: \"c6daea69-8c76-4fba-8960-d0908e4e3b96\") "
Jul 27 21:57:39.540287 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.540262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca" (OuterVolumeSpecName: "service-ca") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:57:39.540409 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.540383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:57:39.542302 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.542280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7" (OuterVolumeSpecName: "kube-api-access-cf5q7") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "kube-api-access-cf5q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:57:39.542302 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.542284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:57:39.542427 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.542309    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:57:39.544315 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.544293    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config" (OuterVolumeSpecName: "console-config") pod "c6daea69-8c76-4fba-8960-d0908e4e3b96" (UID: "c6daea69-8c76-4fba-8960-d0908e4e3b96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:57:39.640603 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640574    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-oauth-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:39.640603 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640599    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:39.640603 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640608    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:39.640842 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640617    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-oauth-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:39.640842 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640626    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf5q7\" (UniqueName: \"kubernetes.io/projected/c6daea69-8c76-4fba-8960-d0908e4e3b96-kube-api-access-cf5q7\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:39.640842 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:39.640636    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6daea69-8c76-4fba-8960-d0908e4e3b96-console-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:57:40.202829 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.202789    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6daea69-8c76-4fba-8960-d0908e4e3b96" containerID="c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa" exitCode=0
Jul 27 21:57:40.203241 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.202838    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5584bcbc7-54964"
Jul 27 21:57:40.203241 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.202878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5584bcbc7-54964" event={"ID":"c6daea69-8c76-4fba-8960-d0908e4e3b96","Type":"ContainerDied","Data":"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"}
Jul 27 21:57:40.203241 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.202936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5584bcbc7-54964" event={"ID":"c6daea69-8c76-4fba-8960-d0908e4e3b96","Type":"ContainerDied","Data":"d4fadbd9517677c5d26af5a9db9564cc2d4fb692aa2651eda241025f94e8664a"}
Jul 27 21:57:40.203241 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.202959    2578 scope.go:117] "RemoveContainer" containerID="c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"
Jul 27 21:57:40.211769 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.211752    2578 scope.go:117] "RemoveContainer" containerID="c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"
Jul 27 21:57:40.212040 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:57:40.212018    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa\": container with ID starting with c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa not found: ID does not exist" containerID="c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"
Jul 27 21:57:40.212088 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.212049    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa"} err="failed to get container status \"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa\": rpc error: code = NotFound desc = could not find container \"c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa\": container with ID starting with c73222658bf6ab3067b9a20e4dcbf44d53af3875a2bf81c1d52331bb15d8d7fa not found: ID does not exist"
Jul 27 21:57:40.222962 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.222942    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:57:40.226120 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.226102    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5584bcbc7-54964"]
Jul 27 21:57:40.321630 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:57:40.321602    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6daea69-8c76-4fba-8960-d0908e4e3b96" path="/var/lib/kubelet/pods/c6daea69-8c76-4fba-8960-d0908e4e3b96/volumes"
Jul 27 21:58:14.453042 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.453003    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 21:58:14.453592 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.453274    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6daea69-8c76-4fba-8960-d0908e4e3b96" containerName="console"
Jul 27 21:58:14.453592 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.453286    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6daea69-8c76-4fba-8960-d0908e4e3b96" containerName="console"
Jul 27 21:58:14.453592 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.453345    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6daea69-8c76-4fba-8960-d0908e4e3b96" containerName="console"
Jul 27 21:58:14.454979 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.454960    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.476303 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.476283    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 21:58:14.522644 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522617    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522805 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522948 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62pk6\" (UniqueName: \"kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522948 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.522948 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.522894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624025 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.623996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624025 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62pk6\" (UniqueName: \"kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624208 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624314 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624314 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624417 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624819 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624923 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.624923 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.625009 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.624959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.626876 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.626851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.627121 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.627100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.632551 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.632528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62pk6\" (UniqueName: \"kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6\") pod \"console-b6bdbfcd-8lfrc\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") " pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.765351 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.765319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:14.898776 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:14.898751    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 21:58:14.901053 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:58:14.901028    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf081ec_f378_4c7c_854e_c7976f527508.slice/crio-aee7ea14b444d3ca8142aeb850541c8d513754df232f4dcaac6e45fe16eb63e9 WatchSource:0}: Error finding container aee7ea14b444d3ca8142aeb850541c8d513754df232f4dcaac6e45fe16eb63e9: Status 404 returned error can't find the container with id aee7ea14b444d3ca8142aeb850541c8d513754df232f4dcaac6e45fe16eb63e9
Jul 27 21:58:15.333086 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:15.333055    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6bdbfcd-8lfrc" event={"ID":"fbf081ec-f378-4c7c-854e-c7976f527508","Type":"ContainerStarted","Data":"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"}
Jul 27 21:58:15.333246 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:15.333107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6bdbfcd-8lfrc" event={"ID":"fbf081ec-f378-4c7c-854e-c7976f527508","Type":"ContainerStarted","Data":"aee7ea14b444d3ca8142aeb850541c8d513754df232f4dcaac6e45fe16eb63e9"}
Jul 27 21:58:15.352908 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:15.352859    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b6bdbfcd-8lfrc" podStartSLOduration=1.352843184 podStartE2EDuration="1.352843184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:58:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:58:15.351835562 +0000 UTC m=+205.538676220" watchObservedRunningTime="2026-07-27 21:58:15.352843184 +0000 UTC m=+205.539683843"
Jul 27 21:58:24.766091 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:24.766055    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:24.766600 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:24.766129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:24.770491 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:24.770472    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:25.374319 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:25.374293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 21:58:25.418796 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:25.418768    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:58:50.442072 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.442008    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-649bd87974-2c5dj" podUID="a6b6de28-d954-4c72-9449-467fabfaa6f9" containerName="console" containerID="cri-o://40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576" gracePeriod=15
Jul 27 21:58:50.682035 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.682010    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:58:50.811473 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811657 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811657 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811508    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811657 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811657 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811882 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811719    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhvhx\" (UniqueName: \"kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.811882 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config\") pod \"a6b6de28-d954-4c72-9449-467fabfaa6f9\" (UID: \"a6b6de28-d954-4c72-9449-467fabfaa6f9\") "
Jul 27 21:58:50.812011 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.811991    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:58:50.812082 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.812006    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:58:50.812122 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.812095    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:58:50.812306 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.812290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config" (OuterVolumeSpecName: "console-config") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:58:50.813930 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.813906    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx" (OuterVolumeSpecName: "kube-api-access-rhvhx") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "kube-api-access-rhvhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:58:50.814280 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.814258    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:58:50.814377 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.814283    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a6b6de28-d954-4c72-9449-467fabfaa6f9" (UID: "a6b6de28-d954-4c72-9449-467fabfaa6f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:58:50.913338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913308    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913335    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-trusted-ca-bundle\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913338 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913346    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhvhx\" (UniqueName: \"kubernetes.io/projected/a6b6de28-d954-4c72-9449-467fabfaa6f9-kube-api-access-rhvhx\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913356    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913364    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913372    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b6de28-d954-4c72-9449-467fabfaa6f9-oauth-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:50.913606 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:50.913381    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b6de28-d954-4c72-9449-467fabfaa6f9-console-oauth-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 21:58:51.471246 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.471210    2578 generic.go:356] "Generic (PLEG): container finished" podID="a6b6de28-d954-4c72-9449-467fabfaa6f9" containerID="40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576" exitCode=0
Jul 27 21:58:51.471768 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.471265    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-649bd87974-2c5dj"
Jul 27 21:58:51.471768 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.471296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649bd87974-2c5dj" event={"ID":"a6b6de28-d954-4c72-9449-467fabfaa6f9","Type":"ContainerDied","Data":"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"}
Jul 27 21:58:51.471768 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.471346    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-649bd87974-2c5dj" event={"ID":"a6b6de28-d954-4c72-9449-467fabfaa6f9","Type":"ContainerDied","Data":"20a72d38536c1146a29c07d4a6e4691ef86d382535d5b4c53344bb24a7433c0a"}
Jul 27 21:58:51.471768 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.471366    2578 scope.go:117] "RemoveContainer" containerID="40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"
Jul 27 21:58:51.480195 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.480179    2578 scope.go:117] "RemoveContainer" containerID="40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"
Jul 27 21:58:51.480431 ip-10-0-136-91 kubenswrapper[2578]: E0727 21:58:51.480414    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576\": container with ID starting with 40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576 not found: ID does not exist" containerID="40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"
Jul 27 21:58:51.480494 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.480445    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576"} err="failed to get container status \"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576\": rpc error: code = NotFound desc = could not find container \"40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576\": container with ID starting with 40e362ed3bb1f2a6178b681f2cd0fd5aa062f5037b66c206db13c5e7750b4576 not found: ID does not exist"
Jul 27 21:58:51.495043 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.495015    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:58:51.504406 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:51.504363    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-649bd87974-2c5dj"]
Jul 27 21:58:52.321820 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:58:52.321786    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b6de28-d954-4c72-9449-467fabfaa6f9" path="/var/lib/kubelet/pods/a6b6de28-d954-4c72-9449-467fabfaa6f9/volumes"
Jul 27 21:59:19.040977 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.040899    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx8fj"]
Jul 27 21:59:19.041472 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.041185    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6b6de28-d954-4c72-9449-467fabfaa6f9" containerName="console"
Jul 27 21:59:19.041472 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.041199    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6b6de28-d954-4c72-9449-467fabfaa6f9" containerName="console"
Jul 27 21:59:19.041472 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.041260    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6b6de28-d954-4c72-9449-467fabfaa6f9" containerName="console"
Jul 27 21:59:19.042979 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.042963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.045639 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.045614    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 27 21:59:19.046568 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.046550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 27 21:59:19.046961 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.046942    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mc45b\""
Jul 27 21:59:19.056325 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.056303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx8fj"]
Jul 27 21:59:19.131367 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.131324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.131367 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.131377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlhbc\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-kube-api-access-nlhbc\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.232646 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.232611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.232831 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.232656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlhbc\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-kube-api-access-nlhbc\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.251454 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.251421    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.251635 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.251613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlhbc\" (UniqueName: \"kubernetes.io/projected/3311d646-a8ed-4387-aa74-ac6566d106e0-kube-api-access-nlhbc\") pod \"cert-manager-webhook-6486ffd997-nx8fj\" (UID: \"3311d646-a8ed-4387-aa74-ac6566d106e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.352453 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.352355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:19.482594 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.482552    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nx8fj"]
Jul 27 21:59:19.485419 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:59:19.485389    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3311d646_a8ed_4387_aa74_ac6566d106e0.slice/crio-711e9e45f08436e23e9636b262f899c631c7568dd34f7793320efaad10f6328d WatchSource:0}: Error finding container 711e9e45f08436e23e9636b262f899c631c7568dd34f7793320efaad10f6328d: Status 404 returned error can't find the container with id 711e9e45f08436e23e9636b262f899c631c7568dd34f7793320efaad10f6328d
Jul 27 21:59:19.577893 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:19.577862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj" event={"ID":"3311d646-a8ed-4387-aa74-ac6566d106e0","Type":"ContainerStarted","Data":"711e9e45f08436e23e9636b262f899c631c7568dd34f7793320efaad10f6328d"}
Jul 27 21:59:20.837273 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.837241    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"]
Jul 27 21:59:20.839854 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.839782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.842432 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.842407    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fjtvr\""
Jul 27 21:59:20.843963 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.843935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9v7s\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-kube-api-access-l9v7s\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.844133 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.843995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.850315 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.850291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"]
Jul 27 21:59:20.944855 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.944816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9v7s\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-kube-api-access-l9v7s\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.945034 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.944883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.954147 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.954091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9v7s\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-kube-api-access-l9v7s\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:20.954345 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:20.954321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6cc19681-6aa6-4eed-a391-ba9e89483784-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f2zf4\" (UID: \"6cc19681-6aa6-4eed-a391-ba9e89483784\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:21.154136 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:21.154043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"
Jul 27 21:59:21.319957 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:21.319928    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4"]
Jul 27 21:59:21.323071 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:59:21.323035    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cc19681_6aa6_4eed_a391_ba9e89483784.slice/crio-bd57d266d970334b8180fc14703300c94fc22673dfc97277492f29b07dcdfc00 WatchSource:0}: Error finding container bd57d266d970334b8180fc14703300c94fc22673dfc97277492f29b07dcdfc00: Status 404 returned error can't find the container with id bd57d266d970334b8180fc14703300c94fc22673dfc97277492f29b07dcdfc00
Jul 27 21:59:21.588582 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:21.588544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4" event={"ID":"6cc19681-6aa6-4eed-a391-ba9e89483784","Type":"ContainerStarted","Data":"bd57d266d970334b8180fc14703300c94fc22673dfc97277492f29b07dcdfc00"}
Jul 27 21:59:23.603132 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:23.603088    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj" event={"ID":"3311d646-a8ed-4387-aa74-ac6566d106e0","Type":"ContainerStarted","Data":"8b332fad0086003da5a66c0e8da4e110dc9c96a3143c9e3dd9cadc19ae9e2ffe"}
Jul 27 21:59:23.603614 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:23.603150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:23.604706 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:23.604657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4" event={"ID":"6cc19681-6aa6-4eed-a391-ba9e89483784","Type":"ContainerStarted","Data":"60e164ee1edc8728aa7f42b1e196b1383d4919095ecd2a7d702a675c9a1c861f"}
Jul 27 21:59:23.633825 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:23.633787    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj" podStartSLOduration=1.011079327 podStartE2EDuration="4.633773211s" totalImagesPullingTime="3.622693884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:59:23.631195407 +0000 UTC m=+273.818036066" watchObservedRunningTime="2026-07-27 21:59:23.633773211 +0000 UTC m=+273.820613871"
Jul 27 21:59:23.658897 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:23.658856    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f2zf4" podStartSLOduration=1.877431396 podStartE2EDuration="3.658841088s" totalImagesPullingTime="1.781409692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:59:23.657953295 +0000 UTC m=+273.844793967" watchObservedRunningTime="2026-07-27 21:59:23.658841088 +0000 UTC m=+273.845681746"
Jul 27 21:59:29.610125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:29.610099    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nx8fj"
Jul 27 21:59:50.140553 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:50.140524    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 21:59:50.143942 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:50.143921    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 21:59:50.151814 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:50.151779    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 21:59:55.266827 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.266795    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"]
Jul 27 21:59:55.270420 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.270398    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.274223 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.274202    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 21:59:55.275125 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.275107    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 21:59:55.275191 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.275111    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 21:59:55.276381 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.276362    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pt8cs\""
Jul 27 21:59:55.279296 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.279279    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 21:59:55.312462 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.312440    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"]
Jul 27 21:59:55.411830 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.411801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-manifests\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.411974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.411835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.411974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.411861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwwrt\" (UniqueName: \"kubernetes.io/projected/c3737160-6d9d-4577-b5ff-2d820e854902-kube-api-access-dwwrt\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.411974 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.411954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-tmp\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.412101 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.411998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513055 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-manifests\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513181 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513181 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwwrt\" (UniqueName: \"kubernetes.io/projected/c3737160-6d9d-4577-b5ff-2d820e854902-kube-api-access-dwwrt\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513292 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-tmp\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513292 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.513475 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.513457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-manifests\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.515526 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.515500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3737160-6d9d-4577-b5ff-2d820e854902-tmp\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.515783 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.515766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.515838 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.515786    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3737160-6d9d-4577-b5ff-2d820e854902-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.522833 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.522784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwwrt\" (UniqueName: \"kubernetes.io/projected/c3737160-6d9d-4577-b5ff-2d820e854902-kube-api-access-dwwrt\") pod \"opendatahub-operator-controller-manager-5f8fb7b654-rp2mj\" (UID: \"c3737160-6d9d-4577-b5ff-2d820e854902\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.580932 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.580907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 21:59:55.716619 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.716590    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"]
Jul 27 21:59:55.720496 ip-10-0-136-91 kubenswrapper[2578]: W0727 21:59:55.720461    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3737160_6d9d_4577_b5ff_2d820e854902.slice/crio-6d03bc2d92316f25e5db1877c417f55a8cf15664a8c412cf07a780dbcd235839 WatchSource:0}: Error finding container 6d03bc2d92316f25e5db1877c417f55a8cf15664a8c412cf07a780dbcd235839: Status 404 returned error can't find the container with id 6d03bc2d92316f25e5db1877c417f55a8cf15664a8c412cf07a780dbcd235839
Jul 27 21:59:55.722339 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.722323    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:59:55.729157 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:55.729133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj" event={"ID":"c3737160-6d9d-4577-b5ff-2d820e854902","Type":"ContainerStarted","Data":"6d03bc2d92316f25e5db1877c417f55a8cf15664a8c412cf07a780dbcd235839"}
Jul 27 21:59:59.748316 ip-10-0-136-91 kubenswrapper[2578]: I0727 21:59:59.748280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj" event={"ID":"c3737160-6d9d-4577-b5ff-2d820e854902","Type":"ContainerStarted","Data":"7f1bff941f02bc73bae84260db226d588c3e61cf658d8f2ece9ec2559ba3f8e7"}
Jul 27 22:00:00.753634 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:00.753600    2578 generic.go:356] "Generic (PLEG): container finished" podID="c3737160-6d9d-4577-b5ff-2d820e854902" containerID="7f1bff941f02bc73bae84260db226d588c3e61cf658d8f2ece9ec2559ba3f8e7" exitCode=0
Jul 27 22:00:00.754036 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:00.753706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj" event={"ID":"c3737160-6d9d-4577-b5ff-2d820e854902","Type":"ContainerDied","Data":"7f1bff941f02bc73bae84260db226d588c3e61cf658d8f2ece9ec2559ba3f8e7"}
Jul 27 22:00:01.759890 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:01.759809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj" event={"ID":"c3737160-6d9d-4577-b5ff-2d820e854902","Type":"ContainerStarted","Data":"f2526cf18c20c977011b2a5776627b6184d56f16952a837ef9bfcf164249798c"}
Jul 27 22:00:01.760259 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:01.759908    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 22:00:01.794391 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:01.794331    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj" podStartSLOduration=2.040193282 podStartE2EDuration="6.79431253s" totalImagesPullingTime="3.732516078s" totalInitContainerRuntime="1.02160317s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:59:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:01.793732073 +0000 UTC m=+311.980572732" watchObservedRunningTime="2026-07-27 22:00:01.79431253 +0000 UTC m=+311.981153191"
Jul 27 22:00:12.766766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:12.766735    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f8fb7b654-rp2mj"
Jul 27 22:00:15.073406 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.073365    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"]
Jul 27 22:00:15.075503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.075481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.079026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.078998    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6m4c7\""
Jul 27 22:00:15.079130 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.079025    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 22:00:15.079130 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.079026    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 22:00:15.079326 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.079310    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 22:00:15.079398 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.079377    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 22:00:15.136851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.136822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"]
Jul 27 22:00:15.179424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.179401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.179554 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.179438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdmj\" (UniqueName: \"kubernetes.io/projected/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-kube-api-access-zbdmj\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.179554 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.179481    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tmp\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.280376 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.280348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.280511 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.280390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdmj\" (UniqueName: \"kubernetes.io/projected/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-kube-api-access-zbdmj\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.280511 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.280428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tmp\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.280511 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:15.280500    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 27 22:00:15.280608 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:15.280601    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs podName:7e5f098d-b052-40cf-9bec-7ffd1f7d1191 nodeName:}" failed. No retries permitted until 2026-07-27 22:00:15.780580351 +0000 UTC m=+325.967420988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs") pod "kube-auth-proxy-5fcc8648dc-hwn9f" (UID: "7e5f098d-b052-40cf-9bec-7ffd1f7d1191") : secret "kube-auth-proxy-tls" not found
Jul 27 22:00:15.282891 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.282869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tmp\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.293993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.293968    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdmj\" (UniqueName: \"kubernetes.io/projected/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-kube-api-access-zbdmj\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.784318 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.784284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.786836 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.786819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f098d-b052-40cf-9bec-7ffd1f7d1191-tls-certs\") pod \"kube-auth-proxy-5fcc8648dc-hwn9f\" (UID: \"7e5f098d-b052-40cf-9bec-7ffd1f7d1191\") " pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:15.985391 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:15.985356    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"
Jul 27 22:00:16.127057 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.127024    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f"]
Jul 27 22:00:16.129468 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:00:16.129433    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5f098d_b052_40cf_9bec_7ffd1f7d1191.slice/crio-9777025e7a25641c1c632f7006613f7ac803390222785e3031b8ea776696c888 WatchSource:0}: Error finding container 9777025e7a25641c1c632f7006613f7ac803390222785e3031b8ea776696c888: Status 404 returned error can't find the container with id 9777025e7a25641c1c632f7006613f7ac803390222785e3031b8ea776696c888
Jul 27 22:00:16.521901 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.521870    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:16.524741 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.524724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.528657 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.528636    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 22:00:16.529312 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.529294    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 22:00:16.529423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.529348    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 22:00:16.529423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.529376    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 22:00:16.529423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.529392    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5g57t\""
Jul 27 22:00:16.540795 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.540775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:16.691883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.691854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.691883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.691891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.692134 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.691916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.692134 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.692050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bmcp\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.692134 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.692088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.692272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.692132    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.692272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.692164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.792841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bmcp\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.792841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.792983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.793809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.793787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.795917 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.795893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.796024 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.796004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.796078 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.796058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.796264 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.796242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.801585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.801553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bmcp\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.801804 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.801784    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-6smdk\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.828336 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.828305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f" event={"ID":"7e5f098d-b052-40cf-9bec-7ffd1f7d1191","Type":"ContainerStarted","Data":"9777025e7a25641c1c632f7006613f7ac803390222785e3031b8ea776696c888"}
Jul 27 22:00:16.834513 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.834491    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:16.991133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:16.990942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:16.993852 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:00:16.993818    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa37c163_a97f_492e_8bdf_2159230db4e4.slice/crio-a080ce20c27003b6b605a82795b392852d87c4cee98f07bc8afe8e63e44fa714 WatchSource:0}: Error finding container a080ce20c27003b6b605a82795b392852d87c4cee98f07bc8afe8e63e44fa714: Status 404 returned error can't find the container with id a080ce20c27003b6b605a82795b392852d87c4cee98f07bc8afe8e63e44fa714
Jul 27 22:00:17.834179 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:17.834141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" event={"ID":"aa37c163-a97f-492e-8bdf-2159230db4e4","Type":"ContainerStarted","Data":"a080ce20c27003b6b605a82795b392852d87c4cee98f07bc8afe8e63e44fa714"}
Jul 27 22:00:19.820428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:19.820393    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:00:19.820815 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:19.820455    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 27 22:00:20.849648 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.849608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f" event={"ID":"7e5f098d-b052-40cf-9bec-7ffd1f7d1191","Type":"ContainerStarted","Data":"3abe6650113b6e10756af0ccf547531288d3dc05935334a28b17ca0c52b80d62"}
Jul 27 22:00:20.851223 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.851201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" event={"ID":"aa37c163-a97f-492e-8bdf-2159230db4e4","Type":"ContainerStarted","Data":"991367fcaf957632a7063f43139e29bbde49e881af27ac41ba2551ba16c32f34"}
Jul 27 22:00:20.851328 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.851240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:20.852993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.852972    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-6smdk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 22:00:20.853124 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.853011    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 22:00:20.874979 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:20.872708    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fcc8648dc-hwn9f" podStartSLOduration=2.210945243 podStartE2EDuration="5.87269064s" totalImagesPullingTime="3.661745397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:20.868429011 +0000 UTC m=+331.055269670" watchObservedRunningTime="2026-07-27 22:00:20.87269064 +0000 UTC m=+331.059531298"
Jul 27 22:00:21.856928 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:21.856898    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:21.878402 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:21.878350    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" podStartSLOduration=3.054379567 podStartE2EDuration="5.878329898s" totalImagesPullingTime="2.823950331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:20.918447575 +0000 UTC m=+331.105288234" watchObservedRunningTime="2026-07-27 22:00:21.878329898 +0000 UTC m=+332.065170558"
Jul 27 22:00:26.813945 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.813911    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"]
Jul 27 22:00:26.815973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.815958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.818980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.818959    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 27 22:00:26.819703 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.819687    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-62xfs\""
Jul 27 22:00:26.824779 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.824757    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"]
Jul 27 22:00:26.826777 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.826761    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:26.829324 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.829307    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 27 22:00:26.829420 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.829400    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-j8g6h\""
Jul 27 22:00:26.836301 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.836273    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"]
Jul 27 22:00:26.851573 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.851551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"]
Jul 27 22:00:26.875816 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.875789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nw86\" (UniqueName: \"kubernetes.io/projected/f0922773-e30f-41fd-a880-625e4617cce8-kube-api-access-8nw86\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:26.875816 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.875819    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjfn\" (UniqueName: \"kubernetes.io/projected/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-kube-api-access-jcjfn\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.875995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.875883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:26.875995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.875946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.976336 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.976307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.976480 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.976362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nw86\" (UniqueName: \"kubernetes.io/projected/f0922773-e30f-41fd-a880-625e4617cce8-kube-api-access-8nw86\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:26.976480 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:26.976455    2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 27 22:00:26.976577 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:26.976521    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert podName:01aaad7e-1e22-46b0-9ade-fa5bdc9fca03 nodeName:}" failed. No retries permitted until 2026-07-27 22:00:27.476505614 +0000 UTC m=+337.663346250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert") pod "kserve-controller-manager-d6f4b65b8-z4rtz" (UID: "01aaad7e-1e22-46b0-9ade-fa5bdc9fca03") : secret "kserve-webhook-server-cert" not found
Jul 27 22:00:26.976730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.976706    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjfn\" (UniqueName: \"kubernetes.io/projected/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-kube-api-access-jcjfn\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.976813 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.976800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:26.976936 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:26.976922    2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 27 22:00:26.976995 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:00:26.976986    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert podName:f0922773-e30f-41fd-a880-625e4617cce8 nodeName:}" failed. No retries permitted until 2026-07-27 22:00:27.476970669 +0000 UTC m=+337.663811306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert") pod "llmisvc-controller-manager-777c6bd7d5-gqvvs" (UID: "f0922773-e30f-41fd-a880-625e4617cce8") : secret "llmisvc-webhook-server-cert" not found
Jul 27 22:00:26.987663 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.987638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjfn\" (UniqueName: \"kubernetes.io/projected/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-kube-api-access-jcjfn\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:26.987953 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:26.987933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nw86\" (UniqueName: \"kubernetes.io/projected/f0922773-e30f-41fd-a880-625e4617cce8-kube-api-access-8nw86\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:27.480529 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.480492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:27.480758 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.480585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:27.483029 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.483007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01aaad7e-1e22-46b0-9ade-fa5bdc9fca03-cert\") pod \"kserve-controller-manager-d6f4b65b8-z4rtz\" (UID: \"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:27.483125 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.483007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0922773-e30f-41fd-a880-625e4617cce8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gqvvs\" (UID: \"f0922773-e30f-41fd-a880-625e4617cce8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:27.726039 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.726003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:27.736415 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.736389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:27.890114 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.890046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"]
Jul 27 22:00:27.893864 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:00:27.893839    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01aaad7e_1e22_46b0_9ade_fa5bdc9fca03.slice/crio-128573234bb9ad3222f65d37d80450b4e3c6a1da9eee5ff574296f3fc3960a91 WatchSource:0}: Error finding container 128573234bb9ad3222f65d37d80450b4e3c6a1da9eee5ff574296f3fc3960a91: Status 404 returned error can't find the container with id 128573234bb9ad3222f65d37d80450b4e3c6a1da9eee5ff574296f3fc3960a91
Jul 27 22:00:27.923655 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:27.923631    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"]
Jul 27 22:00:27.925271 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:00:27.925246    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0922773_e30f_41fd_a880_625e4617cce8.slice/crio-5a105f744503c8d92f558d9f0aa3fbbfc848ea38fea1c43b4253311ca7816adc WatchSource:0}: Error finding container 5a105f744503c8d92f558d9f0aa3fbbfc848ea38fea1c43b4253311ca7816adc: Status 404 returned error can't find the container with id 5a105f744503c8d92f558d9f0aa3fbbfc848ea38fea1c43b4253311ca7816adc
Jul 27 22:00:28.891276 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:28.891240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs" event={"ID":"f0922773-e30f-41fd-a880-625e4617cce8","Type":"ContainerStarted","Data":"5a105f744503c8d92f558d9f0aa3fbbfc848ea38fea1c43b4253311ca7816adc"}
Jul 27 22:00:28.892396 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:28.892375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz" event={"ID":"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03","Type":"ContainerStarted","Data":"128573234bb9ad3222f65d37d80450b4e3c6a1da9eee5ff574296f3fc3960a91"}
Jul 27 22:00:31.477582 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.477483    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:31.478044 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.477765    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerName="discovery" containerID="cri-o://991367fcaf957632a7063f43139e29bbde49e881af27ac41ba2551ba16c32f34" gracePeriod=30
Jul 27 22:00:31.909812 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.909772    2578 generic.go:356] "Generic (PLEG): container finished" podID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerID="991367fcaf957632a7063f43139e29bbde49e881af27ac41ba2551ba16c32f34" exitCode=0
Jul 27 22:00:31.909995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.909848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" event={"ID":"aa37c163-a97f-492e-8bdf-2159230db4e4","Type":"ContainerDied","Data":"991367fcaf957632a7063f43139e29bbde49e881af27ac41ba2551ba16c32f34"}
Jul 27 22:00:31.911535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.911511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz" event={"ID":"01aaad7e-1e22-46b0-9ade-fa5bdc9fca03","Type":"ContainerStarted","Data":"e82d34809f15a34cf02862b69828a98454184526c39d5fa257dfc759c5704f7a"}
Jul 27 22:00:31.911743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.911715    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:00:31.930809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:31.930753    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz" podStartSLOduration=2.705848012 podStartE2EDuration="5.93073552s" totalImagesPullingTime="3.224887508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:31.929884216 +0000 UTC m=+342.116724876" watchObservedRunningTime="2026-07-27 22:00:31.93073552 +0000 UTC m=+342.117576181"
Jul 27 22:00:32.251045 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.251016    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:32.326565 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326572    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326612    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326646    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.326744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.326738    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bmcp\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp\") pod \"aa37c163-a97f-492e-8bdf-2159230db4e4\" (UID: \"aa37c163-a97f-492e-8bdf-2159230db4e4\") "
Jul 27 22:00:32.327142 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.327117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:00:32.329392 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329353    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:00:32.329510 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329392    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp" (OuterVolumeSpecName: "kube-api-access-5bmcp") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "kube-api-access-5bmcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:00:32.329510 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329488    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:00:32.329633 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:00:32.329871 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329850    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts" (OuterVolumeSpecName: "cacerts") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:00:32.329958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.329876    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs" (OuterVolumeSpecName: "local-certs") pod "aa37c163-a97f-492e-8bdf-2159230db4e4" (UID: "aa37c163-a97f-492e-8bdf-2159230db4e4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:00:32.427813 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427776    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa37c163-a97f-492e-8bdf-2159230db4e4-local-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427810    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-token\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427855    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-ca-configmap\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427869    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-csr-dns-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427882    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-istio-kubeconfig\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427894    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/aa37c163-a97f-492e-8bdf-2159230db4e4-cacerts\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.428026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.427906    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bmcp\" (UniqueName: \"kubernetes.io/projected/aa37c163-a97f-492e-8bdf-2159230db4e4-kube-api-access-5bmcp\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:00:32.916714 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.916657    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"
Jul 27 22:00:32.916714 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.916702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk" event={"ID":"aa37c163-a97f-492e-8bdf-2159230db4e4","Type":"ContainerDied","Data":"a080ce20c27003b6b605a82795b392852d87c4cee98f07bc8afe8e63e44fa714"}
Jul 27 22:00:32.917221 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.916766    2578 scope.go:117] "RemoveContainer" containerID="991367fcaf957632a7063f43139e29bbde49e881af27ac41ba2551ba16c32f34"
Jul 27 22:00:32.918344 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.918318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs" event={"ID":"f0922773-e30f-41fd-a880-625e4617cce8","Type":"ContainerStarted","Data":"38fd57bbd5f038aff972ebbd17b7d7105ebb2ad8b0dd369d3397e9fa342ef3da"}
Jul 27 22:00:32.918475 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.918403    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:00:32.940400 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.940338    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs" podStartSLOduration=2.60408816 podStartE2EDuration="6.940322333s" totalImagesPullingTime="4.336234173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:00:32.939531142 +0000 UTC m=+343.126371798" watchObservedRunningTime="2026-07-27 22:00:32.940322333 +0000 UTC m=+343.127162992"
Jul 27 22:00:32.953993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.953963    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:32.958760 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:32.958739    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-6smdk"]
Jul 27 22:00:34.322703 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:00:34.322652    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" path="/var/lib/kubelet/pods/aa37c163-a97f-492e-8bdf-2159230db4e4/volumes"
Jul 27 22:01:02.923830 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:02.923800    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z4rtz"
Jul 27 22:01:03.925187 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:03.925159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gqvvs"
Jul 27 22:01:16.048421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.048387    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:16.048885 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.048731    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerName="discovery"
Jul 27 22:01:16.048885 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.048745    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerName="discovery"
Jul 27 22:01:16.048885 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.048829    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa37c163-a97f-492e-8bdf-2159230db4e4" containerName="discovery"
Jul 27 22:01:16.050810 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.050792    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:16.053414 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.053395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 22:01:16.054359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.054344    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 22:01:16.054418 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.054346    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-r6xl9\""
Jul 27 22:01:16.060096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.060072    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:16.089718 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.089693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlk24\" (UniqueName: \"kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24\") pod \"kuadrant-operator-catalog-xxjvn\" (UID: \"022cfe85-ba5c-4277-a167-06d097717029\") " pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:16.190814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.190781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlk24\" (UniqueName: \"kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24\") pod \"kuadrant-operator-catalog-xxjvn\" (UID: \"022cfe85-ba5c-4277-a167-06d097717029\") " pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:16.198815 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.198787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlk24\" (UniqueName: \"kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24\") pod \"kuadrant-operator-catalog-xxjvn\" (UID: \"022cfe85-ba5c-4277-a167-06d097717029\") " pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:16.361101 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.361026    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:16.410087 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.410060    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:16.487602 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.487579    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:16.489296 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:16.489270    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022cfe85_ba5c_4277_a167_06d097717029.slice/crio-71fef147d94315c571d052ed4c790a5a4c0524b7d6320b0d6878053c49a8f9af WatchSource:0}: Error finding container 71fef147d94315c571d052ed4c790a5a4c0524b7d6320b0d6878053c49a8f9af: Status 404 returned error can't find the container with id 71fef147d94315c571d052ed4c790a5a4c0524b7d6320b0d6878053c49a8f9af
Jul 27 22:01:16.619910 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.619834    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d94sm"]
Jul 27 22:01:16.623058 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.623043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:16.632216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.632193    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d94sm"]
Jul 27 22:01:16.694026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.693986    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7dbz\" (UniqueName: \"kubernetes.io/projected/38284300-bbf6-4875-8cf5-f93a0ac7eba7-kube-api-access-s7dbz\") pod \"kuadrant-operator-catalog-d94sm\" (UID: \"38284300-bbf6-4875-8cf5-f93a0ac7eba7\") " pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:16.794998 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.794966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7dbz\" (UniqueName: \"kubernetes.io/projected/38284300-bbf6-4875-8cf5-f93a0ac7eba7-kube-api-access-s7dbz\") pod \"kuadrant-operator-catalog-d94sm\" (UID: \"38284300-bbf6-4875-8cf5-f93a0ac7eba7\") " pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:16.803210 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.803179    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7dbz\" (UniqueName: \"kubernetes.io/projected/38284300-bbf6-4875-8cf5-f93a0ac7eba7-kube-api-access-s7dbz\") pod \"kuadrant-operator-catalog-d94sm\" (UID: \"38284300-bbf6-4875-8cf5-f93a0ac7eba7\") " pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:16.932879 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:16.932797    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:17.070061 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:17.070037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d94sm"]
Jul 27 22:01:17.072050 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:17.072024    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38284300_bbf6_4875_8cf5_f93a0ac7eba7.slice/crio-82b6852e11ccd7925689932d3f992b2b4b43a2754c37e7b1d73c208d45c7a735 WatchSource:0}: Error finding container 82b6852e11ccd7925689932d3f992b2b4b43a2754c37e7b1d73c208d45c7a735: Status 404 returned error can't find the container with id 82b6852e11ccd7925689932d3f992b2b4b43a2754c37e7b1d73c208d45c7a735
Jul 27 22:01:17.103854 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:17.103822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" event={"ID":"022cfe85-ba5c-4277-a167-06d097717029","Type":"ContainerStarted","Data":"71fef147d94315c571d052ed4c790a5a4c0524b7d6320b0d6878053c49a8f9af"}
Jul 27 22:01:17.104937 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:17.104913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d94sm" event={"ID":"38284300-bbf6-4875-8cf5-f93a0ac7eba7","Type":"ContainerStarted","Data":"82b6852e11ccd7925689932d3f992b2b4b43a2754c37e7b1d73c208d45c7a735"}
Jul 27 22:01:19.117853 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.117816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" event={"ID":"022cfe85-ba5c-4277-a167-06d097717029","Type":"ContainerStarted","Data":"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"}
Jul 27 22:01:19.117853 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.117843    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" podUID="022cfe85-ba5c-4277-a167-06d097717029" containerName="registry-server" containerID="cri-o://a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf" gracePeriod=2
Jul 27 22:01:19.134481 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.134434    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" podStartSLOduration=0.971299131 podStartE2EDuration="3.13441619s" totalImagesPullingTime="2.163117059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:19.133295925 +0000 UTC m=+389.320136586" watchObservedRunningTime="2026-07-27 22:01:19.13441619 +0000 UTC m=+389.321256849"
Jul 27 22:01:19.352405 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.352379    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:19.413658 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.413588    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlk24\" (UniqueName: \"kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24\") pod \"022cfe85-ba5c-4277-a167-06d097717029\" (UID: \"022cfe85-ba5c-4277-a167-06d097717029\") "
Jul 27 22:01:19.415781 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.415760    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24" (OuterVolumeSpecName: "kube-api-access-mlk24") pod "022cfe85-ba5c-4277-a167-06d097717029" (UID: "022cfe85-ba5c-4277-a167-06d097717029"). InnerVolumeSpecName "kube-api-access-mlk24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:01:19.514934 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:19.514915    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlk24\" (UniqueName: \"kubernetes.io/projected/022cfe85-ba5c-4277-a167-06d097717029-kube-api-access-mlk24\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:01:20.123474 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.123437    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d94sm" event={"ID":"38284300-bbf6-4875-8cf5-f93a0ac7eba7","Type":"ContainerStarted","Data":"a0e27a8c364ff07d77fd28e352ee6be2964747315bda93d8e4807393fa3b5a83"}
Jul 27 22:01:20.124632 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.124611    2578 generic.go:356] "Generic (PLEG): container finished" podID="022cfe85-ba5c-4277-a167-06d097717029" containerID="a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf" exitCode=0
Jul 27 22:01:20.124752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.124646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" event={"ID":"022cfe85-ba5c-4277-a167-06d097717029","Type":"ContainerDied","Data":"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"}
Jul 27 22:01:20.124752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.124654    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn"
Jul 27 22:01:20.124752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.124692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xxjvn" event={"ID":"022cfe85-ba5c-4277-a167-06d097717029","Type":"ContainerDied","Data":"71fef147d94315c571d052ed4c790a5a4c0524b7d6320b0d6878053c49a8f9af"}
Jul 27 22:01:20.124752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.124717    2578 scope.go:117] "RemoveContainer" containerID="a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"
Jul 27 22:01:20.135398 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.135373    2578 scope.go:117] "RemoveContainer" containerID="a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"
Jul 27 22:01:20.135820 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:01:20.135799    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf\": container with ID starting with a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf not found: ID does not exist" containerID="a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"
Jul 27 22:01:20.135902 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.135830    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf"} err="failed to get container status \"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf\": rpc error: code = NotFound desc = could not find container \"a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf\": container with ID starting with a8fb291fd4b76f27015e467fda6e961d8501d3b6e24b2fbf8b5875d6c49006bf not found: ID does not exist"
Jul 27 22:01:20.142031 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.141992    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-d94sm" podStartSLOduration=1.704620567 podStartE2EDuration="4.14197959s" totalImagesPullingTime="2.437359023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:20.14076111 +0000 UTC m=+390.327601768" watchObservedRunningTime="2026-07-27 22:01:20.14197959 +0000 UTC m=+390.328820249"
Jul 27 22:01:20.153157 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.153136    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:20.155507 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.155487    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xxjvn"]
Jul 27 22:01:20.327969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:20.327941    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="022cfe85-ba5c-4277-a167-06d097717029" path="/var/lib/kubelet/pods/022cfe85-ba5c-4277-a167-06d097717029/volumes"
Jul 27 22:01:26.933080 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:26.933053    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:26.933548 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:26.933137    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:26.954987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:26.954967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:27.177057 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:27.177032    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-d94sm"
Jul 27 22:01:42.732096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.732061    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zw628"]
Jul 27 22:01:42.732467 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.732366    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="022cfe85-ba5c-4277-a167-06d097717029" containerName="registry-server"
Jul 27 22:01:42.732467 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.732378    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="022cfe85-ba5c-4277-a167-06d097717029" containerName="registry-server"
Jul 27 22:01:42.732467 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.732441    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="022cfe85-ba5c-4277-a167-06d097717029" containerName="registry-server"
Jul 27 22:01:42.735278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.735263    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:42.737931 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.737909    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pjztg\""
Jul 27 22:01:42.747840 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.747822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zw628"]
Jul 27 22:01:42.806122 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.806090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxb2n\" (UniqueName: \"kubernetes.io/projected/6538384e-ba9a-477f-86e9-abca7b938934-kube-api-access-zxb2n\") pod \"authorino-operator-75cc8d5cb7-zw628\" (UID: \"6538384e-ba9a-477f-86e9-abca7b938934\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:42.907568 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.907531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxb2n\" (UniqueName: \"kubernetes.io/projected/6538384e-ba9a-477f-86e9-abca7b938934-kube-api-access-zxb2n\") pod \"authorino-operator-75cc8d5cb7-zw628\" (UID: \"6538384e-ba9a-477f-86e9-abca7b938934\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:42.920640 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:42.920611    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxb2n\" (UniqueName: \"kubernetes.io/projected/6538384e-ba9a-477f-86e9-abca7b938934-kube-api-access-zxb2n\") pod \"authorino-operator-75cc8d5cb7-zw628\" (UID: \"6538384e-ba9a-477f-86e9-abca7b938934\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:43.045693 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:43.045643    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:43.178273 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:43.178249    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zw628"]
Jul 27 22:01:43.180651 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:43.180624    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6538384e_ba9a_477f_86e9_abca7b938934.slice/crio-462aa2c116ad2a979d63ff80027ecf17a32d369bc50ac626c968ad998032bbc8 WatchSource:0}: Error finding container 462aa2c116ad2a979d63ff80027ecf17a32d369bc50ac626c968ad998032bbc8: Status 404 returned error can't find the container with id 462aa2c116ad2a979d63ff80027ecf17a32d369bc50ac626c968ad998032bbc8
Jul 27 22:01:43.225123 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:43.225089    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628" event={"ID":"6538384e-ba9a-477f-86e9-abca7b938934","Type":"ContainerStarted","Data":"462aa2c116ad2a979d63ff80027ecf17a32d369bc50ac626c968ad998032bbc8"}
Jul 27 22:01:46.242403 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.242365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628" event={"ID":"6538384e-ba9a-477f-86e9-abca7b938934","Type":"ContainerStarted","Data":"4266d15cc82cbd9c6a10c570a70e2fe0e333ea36fdc0f580948257e1031755da"}
Jul 27 22:01:46.242819 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.242488    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:46.307233 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.307181    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628" podStartSLOduration=2.143957801 podStartE2EDuration="4.307165654s" totalImagesPullingTime="2.163207853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:46.30708925 +0000 UTC m=+416.493929909" watchObservedRunningTime="2026-07-27 22:01:46.307165654 +0000 UTC m=+416.494006312"
Jul 27 22:01:46.692042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.691969    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"]
Jul 27 22:01:46.695255 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.695240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.697883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.697861    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gb9r2\""
Jul 27 22:01:46.711242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.711212    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"]
Jul 27 22:01:46.844522 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.844484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.844720 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.844600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rtg\" (UniqueName: \"kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.946131 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.946052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rtg\" (UniqueName: \"kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.946257 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.946134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.946512 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.946492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:46.954830 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:46.954794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rtg\" (UniqueName: \"kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:47.005603 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:47.005572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:47.160417 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:47.160298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"]
Jul 27 22:01:47.163789 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:47.163759    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5aea8c_8499_4c14_be06_598466c0a58f.slice/crio-326bf0b5e9a9c1d6073c6a1ee72def8c4d8825ce1125545428c0b69d53ba2658 WatchSource:0}: Error finding container 326bf0b5e9a9c1d6073c6a1ee72def8c4d8825ce1125545428c0b69d53ba2658: Status 404 returned error can't find the container with id 326bf0b5e9a9c1d6073c6a1ee72def8c4d8825ce1125545428c0b69d53ba2658
Jul 27 22:01:47.249725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:47.249655    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v" event={"ID":"ca5aea8c-8499-4c14-be06-598466c0a58f","Type":"ContainerStarted","Data":"326bf0b5e9a9c1d6073c6a1ee72def8c4d8825ce1125545428c0b69d53ba2658"}
Jul 27 22:01:48.849752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.849723    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"]
Jul 27 22:01:48.854350 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.854334    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:48.857033 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.856996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 27 22:01:48.857202 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.857085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-fwbh5\""
Jul 27 22:01:48.866612 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.866591    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"]
Jul 27 22:01:48.962819 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:48.962788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sxkc\" (UniqueName: \"kubernetes.io/projected/357f03da-b554-45dd-aea1-79aafe5c7845-kube-api-access-5sxkc\") pod \"dns-operator-controller-manager-864dd78c4c-s7rnr\" (UID: \"357f03da-b554-45dd-aea1-79aafe5c7845\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:49.064030 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:49.063998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sxkc\" (UniqueName: \"kubernetes.io/projected/357f03da-b554-45dd-aea1-79aafe5c7845-kube-api-access-5sxkc\") pod \"dns-operator-controller-manager-864dd78c4c-s7rnr\" (UID: \"357f03da-b554-45dd-aea1-79aafe5c7845\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:49.072802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:49.072781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sxkc\" (UniqueName: \"kubernetes.io/projected/357f03da-b554-45dd-aea1-79aafe5c7845-kube-api-access-5sxkc\") pod \"dns-operator-controller-manager-864dd78c4c-s7rnr\" (UID: \"357f03da-b554-45dd-aea1-79aafe5c7845\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:49.165544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:49.165456    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:49.295812 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:49.295790    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"]
Jul 27 22:01:49.298132 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:49.298101    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod357f03da_b554_45dd_aea1_79aafe5c7845.slice/crio-e9b16b5c807e21cb039fc3174970866ede42b44d6d4beef4b78a08b1cfdc790d WatchSource:0}: Error finding container e9b16b5c807e21cb039fc3174970866ede42b44d6d4beef4b78a08b1cfdc790d: Status 404 returned error can't find the container with id e9b16b5c807e21cb039fc3174970866ede42b44d6d4beef4b78a08b1cfdc790d
Jul 27 22:01:50.266809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.266756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr" event={"ID":"357f03da-b554-45dd-aea1-79aafe5c7845","Type":"ContainerStarted","Data":"e9b16b5c807e21cb039fc3174970866ede42b44d6d4beef4b78a08b1cfdc790d"}
Jul 27 22:01:50.859735 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.859698    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"]
Jul 27 22:01:50.867755 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.867725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:50.870991 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.870964    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rfmsh\""
Jul 27 22:01:50.878781 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.878489    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"]
Jul 27 22:01:50.983321 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:50.983284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmd87\" (UniqueName: \"kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87\") pod \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" (UID: \"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:51.084012 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:51.083976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmd87\" (UniqueName: \"kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87\") pod \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" (UID: \"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:51.094373 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:51.094341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmd87\" (UniqueName: \"kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87\") pod \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" (UID: \"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:51.182484 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:51.182411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:51.379247 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:51.378884    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 27 22:01:51.479725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:51.479700    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"]
Jul 27 22:01:51.480807 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:51.480786    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6f0056_435e_4b74_9721_bdcbe1d3fea4.slice/crio-7a838a12616ad110a8de413cf0fc5c549a2c886b387bb36c480ed86932bd27fb WatchSource:0}: Error finding container 7a838a12616ad110a8de413cf0fc5c549a2c886b387bb36c480ed86932bd27fb: Status 404 returned error can't find the container with id 7a838a12616ad110a8de413cf0fc5c549a2c886b387bb36c480ed86932bd27fb
Jul 27 22:01:52.281748 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:52.281713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7" event={"ID":"3f6f0056-435e-4b74-9721-bdcbe1d3fea4","Type":"ContainerStarted","Data":"7a838a12616ad110a8de413cf0fc5c549a2c886b387bb36c480ed86932bd27fb"}
Jul 27 22:01:52.284622 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:52.284418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr" event={"ID":"357f03da-b554-45dd-aea1-79aafe5c7845","Type":"ContainerStarted","Data":"6d2d0c6bf77673fc8bc82ff523bebcbad94385f0300f0093ebd64656f4289120"}
Jul 27 22:01:52.284622 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:52.284474    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:01:52.304059 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:52.302271    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr" podStartSLOduration=2.226980391 podStartE2EDuration="4.302249776s" totalImagesPullingTime="2.075269385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:52.301246136 +0000 UTC m=+422.488086797" watchObservedRunningTime="2026-07-27 22:01:52.302249776 +0000 UTC m=+422.489090439"
Jul 27 22:01:55.306956 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.306916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v" event={"ID":"ca5aea8c-8499-4c14-be06-598466c0a58f","Type":"ContainerStarted","Data":"fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8"}
Jul 27 22:01:55.307422 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.307051    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:01:55.308998 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.308974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7" event={"ID":"3f6f0056-435e-4b74-9721-bdcbe1d3fea4","Type":"ContainerStarted","Data":"114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8"}
Jul 27 22:01:55.309119 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.309033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:01:55.330150 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.330113    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v" podStartSLOduration=2.085191581 podStartE2EDuration="9.330099264s" totalImagesPullingTime="7.244907683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:55.328137497 +0000 UTC m=+425.514978155" watchObservedRunningTime="2026-07-27 22:01:55.330099264 +0000 UTC m=+425.516939923"
Jul 27 22:01:55.349749 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.349709    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7" podStartSLOduration=2.433948671 podStartE2EDuration="5.349699502s" totalImagesPullingTime="2.915750831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:55.348089741 +0000 UTC m=+425.534930390" watchObservedRunningTime="2026-07-27 22:01:55.349699502 +0000 UTC m=+425.536540163"
Jul 27 22:01:55.677893 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.677791    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54744bbcb4-8jhp8"]
Jul 27 22:01:55.681549 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.681529    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.695025 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.695000    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54744bbcb4-8jhp8"]
Jul 27 22:01:55.824833 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-oauth-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.824987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.824987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l98v\" (UniqueName: \"kubernetes.io/projected/05f20452-1e79-4faa-a040-3174da6d14e0-kube-api-access-2l98v\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.824987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-trusted-ca-bundle\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.824987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-service-ca\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.824987 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.824940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-oauth-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.825156 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.825020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-console-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926323 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-oauth-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926323 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l98v\" (UniqueName: \"kubernetes.io/projected/05f20452-1e79-4faa-a040-3174da6d14e0-kube-api-access-2l98v\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926366    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-trusted-ca-bundle\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-service-ca\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-oauth-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.926544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.926459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-console-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.927269 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.927243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-service-ca\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.927379 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.927354    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-oauth-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.927470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.927455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-trusted-ca-bundle\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.927503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.927487    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/05f20452-1e79-4faa-a040-3174da6d14e0-console-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.928894 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.928850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-oauth-config\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.929129 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.929113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/05f20452-1e79-4faa-a040-3174da6d14e0-console-serving-cert\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.935038 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.935017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l98v\" (UniqueName: \"kubernetes.io/projected/05f20452-1e79-4faa-a040-3174da6d14e0-kube-api-access-2l98v\") pod \"console-54744bbcb4-8jhp8\" (UID: \"05f20452-1e79-4faa-a040-3174da6d14e0\") " pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:55.994525 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:55.994496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:01:56.139073 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:56.139048    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54744bbcb4-8jhp8"]
Jul 27 22:01:56.141507 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:56.141483    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f20452_1e79_4faa_a040_3174da6d14e0.slice/crio-94380698fc7c95cad9d7e55b827b95ce1eebc781dc88b4813533d350ed2fadae WatchSource:0}: Error finding container 94380698fc7c95cad9d7e55b827b95ce1eebc781dc88b4813533d350ed2fadae: Status 404 returned error can't find the container with id 94380698fc7c95cad9d7e55b827b95ce1eebc781dc88b4813533d350ed2fadae
Jul 27 22:01:56.315812 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:56.315775    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54744bbcb4-8jhp8" event={"ID":"05f20452-1e79-4faa-a040-3174da6d14e0","Type":"ContainerStarted","Data":"9d72bd6db373768a3cf1392f5b8cbebeb37002f62b9d55b29bbd8c38b93e16b0"}
Jul 27 22:01:56.316213 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:56.315832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54744bbcb4-8jhp8" event={"ID":"05f20452-1e79-4faa-a040-3174da6d14e0","Type":"ContainerStarted","Data":"94380698fc7c95cad9d7e55b827b95ce1eebc781dc88b4813533d350ed2fadae"}
Jul 27 22:01:56.351756 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:56.351704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54744bbcb4-8jhp8" podStartSLOduration=1.351688289 podStartE2EDuration="1.351688289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:01:56.349011226 +0000 UTC m=+426.535851884" watchObservedRunningTime="2026-07-27 22:01:56.351688289 +0000 UTC m=+426.538528942"
Jul 27 22:01:57.251554 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.251528    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zw628"
Jul 27 22:01:57.277296 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.277272    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"]
Jul 27 22:01:57.280730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.280715    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.283426 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.283395    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 27 22:01:57.283426 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.283410    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 27 22:01:57.283613 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.283501    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4h78d\""
Jul 27 22:01:57.290154 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.289933    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"]
Jul 27 22:01:57.440133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.440097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1656b79b-ae4e-492c-b6a2-8af32ef633df-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.440562 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.440330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gssk8\" (UniqueName: \"kubernetes.io/projected/1656b79b-ae4e-492c-b6a2-8af32ef633df-kube-api-access-gssk8\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.440562 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.440457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.541266 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.541178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1656b79b-ae4e-492c-b6a2-8af32ef633df-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.541266 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.541256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gssk8\" (UniqueName: \"kubernetes.io/projected/1656b79b-ae4e-492c-b6a2-8af32ef633df-kube-api-access-gssk8\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.541458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.541293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.541458 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:01:57.541403    2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 27 22:01:57.541543 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:01:57.541465    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert podName:1656b79b-ae4e-492c-b6a2-8af32ef633df nodeName:}" failed. No retries permitted until 2026-07-27 22:01:58.041446465 +0000 UTC m=+428.228287102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-j5qjx" (UID: "1656b79b-ae4e-492c-b6a2-8af32ef633df") : secret "plugin-serving-cert" not found
Jul 27 22:01:57.541831 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.541813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1656b79b-ae4e-492c-b6a2-8af32ef633df-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:57.550883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:57.550860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gssk8\" (UniqueName: \"kubernetes.io/projected/1656b79b-ae4e-492c-b6a2-8af32ef633df-kube-api-access-gssk8\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:58.046143 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:58.046106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:58.048551 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:58.048534    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1656b79b-ae4e-492c-b6a2-8af32ef633df-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j5qjx\" (UID: \"1656b79b-ae4e-492c-b6a2-8af32ef633df\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:58.193454 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:58.193425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"
Jul 27 22:01:58.338890 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:58.338808    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx"]
Jul 27 22:01:58.341747 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:01:58.341721    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1656b79b_ae4e_492c_b6a2_8af32ef633df.slice/crio-731e73b75446f22c173c2ae0bc7c9d35edf1967a260ec41638b23bd9a0a45bc0 WatchSource:0}: Error finding container 731e73b75446f22c173c2ae0bc7c9d35edf1967a260ec41638b23bd9a0a45bc0: Status 404 returned error can't find the container with id 731e73b75446f22c173c2ae0bc7c9d35edf1967a260ec41638b23bd9a0a45bc0
Jul 27 22:01:59.332004 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:01:59.331974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx" event={"ID":"1656b79b-ae4e-492c-b6a2-8af32ef633df","Type":"ContainerStarted","Data":"731e73b75446f22c173c2ae0bc7c9d35edf1967a260ec41638b23bd9a0a45bc0"}
Jul 27 22:02:03.291943 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:03.291900    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-s7rnr"
Jul 27 22:02:05.994772 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:05.994734    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:02:05.995235 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:05.994813    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:02:06.000419 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:06.000395    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:02:06.331753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:06.325951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:06.331753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:06.326059    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:06.382227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:06.382193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54744bbcb4-8jhp8"
Jul 27 22:02:06.481544 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:06.481512    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 22:02:07.761487 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:07.761452    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"]
Jul 27 22:02:07.774006 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:07.773971    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:07.777074 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:07.777048    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"]
Jul 27 22:02:07.937477 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:07.937440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvkdp\" (UniqueName: \"kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:07.937663 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:07.937483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.038517 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.038430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvkdp\" (UniqueName: \"kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.038517 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.038473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.038975 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.038952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.050269 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.050237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvkdp\" (UniqueName: \"kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp\") pod \"kuadrant-operator-controller-manager-55467d9f7-54q86\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.088173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.088134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:08.198267 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.198225    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"]
Jul 27 22:02:08.205972 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.205936    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"]
Jul 27 22:02:08.222481 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.222441    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:08.226384 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.226352    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.234221 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.234177    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"]
Jul 27 22:02:08.235413 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.234597    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" containerName="manager" containerID="cri-o://fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8" gracePeriod=2
Jul 27 22:02:08.241224 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.241195    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:08.253431 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.253392    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"]
Jul 27 22:02:08.272318 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.272254    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"]
Jul 27 22:02:08.272567 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:08.272256    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:08.272773 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.272474    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" containerName="manager" containerID="cri-o://114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8" gracePeriod=2
Jul 27 22:02:08.274755 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:08.274727    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:08.291253 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.291171    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"]
Jul 27 22:02:08.299166 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:08.299115    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-ldmb7\": pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:08.304355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304318    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"]
Jul 27 22:02:08.304859 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304838    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" containerName="manager"
Jul 27 22:02:08.304859 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304858    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" containerName="manager"
Jul 27 22:02:08.305042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304890    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" containerName="manager"
Jul 27 22:02:08.305042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304900    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" containerName="manager"
Jul 27 22:02:08.305042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.304997    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" containerName="manager"
Jul 27 22:02:08.305042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.305010    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" containerName="manager"
Jul 27 22:02:08.308164 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.308138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:08.310606 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:08.310568    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:08.327615 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.327558    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"]
Jul 27 22:02:08.341530 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:08.340839    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:08.341530 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.341392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6z56\" (UniqueName: \"kubernetes.io/projected/b38c447c-a4a5-4ba8-a324-3b8582e4d29c-kube-api-access-z6z56\") pod \"limitador-operator-controller-manager-859d7b78bd-6ntp7\" (UID: \"b38c447c-a4a5-4ba8-a324-3b8582e4d29c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:08.341530 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.341466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlz29\" (UniqueName: \"kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.341530 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.341528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.442637 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.442602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlz29\" (UniqueName: \"kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.442862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.442658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.442862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.442763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6z56\" (UniqueName: \"kubernetes.io/projected/b38c447c-a4a5-4ba8-a324-3b8582e4d29c-kube-api-access-z6z56\") pod \"limitador-operator-controller-manager-859d7b78bd-6ntp7\" (UID: \"b38c447c-a4a5-4ba8-a324-3b8582e4d29c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:08.443096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.443068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.455598 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.455574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlz29\" (UniqueName: \"kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29\") pod \"kuadrant-operator-controller-manager-55467d9f7-svkbg\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.455771 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.455709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6z56\" (UniqueName: \"kubernetes.io/projected/b38c447c-a4a5-4ba8-a324-3b8582e4d29c-kube-api-access-z6z56\") pod \"limitador-operator-controller-manager-859d7b78bd-6ntp7\" (UID: \"b38c447c-a4a5-4ba8-a324-3b8582e4d29c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:08.541929 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.541842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:08.650394 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:08.650089    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:20.425088 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.425063    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.427481 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.427437    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.429561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.429539    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.432033 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.432001    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.436073 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.435366    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.457733 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.457696    2578 generic.go:356] "Generic (PLEG): container finished" podID="ca5aea8c-8499-4c14-be06-598466c0a58f" containerID="fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8" exitCode=0
Jul 27 22:02:20.457896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.457817    2578 scope.go:117] "RemoveContainer" containerID="fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8"
Jul 27 22:02:20.457981 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.457965    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.459593 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.459573    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8rtg\" (UniqueName: \"kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg\") pod \"ca5aea8c-8499-4c14-be06-598466c0a58f\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") "
Jul 27 22:02:20.459727 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.459621    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmd87\" (UniqueName: \"kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87\") pod \"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\" (UID: \"3f6f0056-435e-4b74-9721-bdcbe1d3fea4\") "
Jul 27 22:02:20.459951 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.459921    2578 generic.go:356] "Generic (PLEG): container finished" podID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" containerID="114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8" exitCode=0
Jul 27 22:02:20.460112 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.460090    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.460344 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.460312    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.460655 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.460566    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume\") pod \"ca5aea8c-8499-4c14-be06-598466c0a58f\" (UID: \"ca5aea8c-8499-4c14-be06-598466c0a58f\") "
Jul 27 22:02:20.461363 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.461342    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca5aea8c-8499-4c14-be06-598466c0a58f" (UID: "ca5aea8c-8499-4c14-be06-598466c0a58f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:20.462584 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.462527    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.462700 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.462593    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87" (OuterVolumeSpecName: "kube-api-access-bmd87") pod "3f6f0056-435e-4b74-9721-bdcbe1d3fea4" (UID: "3f6f0056-435e-4b74-9721-bdcbe1d3fea4"). InnerVolumeSpecName "kube-api-access-bmd87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:20.462813 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.462784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg" (OuterVolumeSpecName: "kube-api-access-s8rtg") pod "ca5aea8c-8499-4c14-be06-598466c0a58f" (UID: "ca5aea8c-8499-4c14-be06-598466c0a58f"). InnerVolumeSpecName "kube-api-access-s8rtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:20.465937 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.465241    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.470088 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.470061    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.470229 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.470140    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"]
Jul 27 22:02:20.475948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.475484    2578 scope.go:117] "RemoveContainer" containerID="fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8"
Jul 27 22:02:20.475948 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.475867    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8\": container with ID starting with fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8 not found: ID does not exist" containerID="fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8"
Jul 27 22:02:20.475948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.475900    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8"} err="failed to get container status \"fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8\": rpc error: code = NotFound desc = could not find container \"fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8\": container with ID starting with fed8bb799b6858128e0987e68baef8610421fad5f720b1ec1bf0d74891317aa8 not found: ID does not exist"
Jul 27 22:02:20.475948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.475925    2578 scope.go:117] "RemoveContainer" containerID="114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8"
Jul 27 22:02:20.478218 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:02:20.478195    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38c447c_a4a5_4ba8_a324_3b8582e4d29c.slice/crio-3c6a3301cf4ee1e59df2787e009718cb05f8b8bf5aa983034657deaca87197a4 WatchSource:0}: Error finding container 3c6a3301cf4ee1e59df2787e009718cb05f8b8bf5aa983034657deaca87197a4: Status 404 returned error can't find the container with id 3c6a3301cf4ee1e59df2787e009718cb05f8b8bf5aa983034657deaca87197a4
Jul 27 22:02:20.486514 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.486495    2578 scope.go:117] "RemoveContainer" containerID="114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8"
Jul 27 22:02:20.486785 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.486768    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8\": container with ID starting with 114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8 not found: ID does not exist" containerID="114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8"
Jul 27 22:02:20.486848 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.486794    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8"} err="failed to get container status \"114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8\": rpc error: code = NotFound desc = could not find container \"114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8\": container with ID starting with 114e8d7187e1d0542622a5c57f6b3870c8b77a275c087e62dd4376707020a6e8 not found: ID does not exist"
Jul 27 22:02:20.497824 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.497802    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:20.499453 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:02:20.499433    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c530dc_ad29_45e1_9b22_95bf59be455c.slice/crio-3dcf7f6d19c2542c6f5badf50b89b087e9e5246633ebb68198cef22c677cdb0b WatchSource:0}: Error finding container 3dcf7f6d19c2542c6f5badf50b89b087e9e5246633ebb68198cef22c677cdb0b: Status 404 returned error can't find the container with id 3dcf7f6d19c2542c6f5badf50b89b087e9e5246633ebb68198cef22c677cdb0b
Jul 27 22:02:20.561376 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.561356    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8rtg\" (UniqueName: \"kubernetes.io/projected/ca5aea8c-8499-4c14-be06-598466c0a58f-kube-api-access-s8rtg\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:20.561450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.561379    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmd87\" (UniqueName: \"kubernetes.io/projected/3f6f0056-435e-4b74-9721-bdcbe1d3fea4-kube-api-access-bmd87\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:20.561450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:20.561390    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca5aea8c-8499-4c14-be06-598466c0a58f-extensions-socket-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:20.771415 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.771382    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.773466 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.773441    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:20.775267 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.775247    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:20.777088 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:20.777071    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:21.467089 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.467054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" event={"ID":"e9c530dc-ad29-45e1-9b22-95bf59be455c","Type":"ContainerStarted","Data":"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"}
Jul 27 22:02:21.467557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.467134    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" event={"ID":"e9c530dc-ad29-45e1-9b22-95bf59be455c","Type":"ContainerStarted","Data":"3dcf7f6d19c2542c6f5badf50b89b087e9e5246633ebb68198cef22c677cdb0b"}
Jul 27 22:02:21.467557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.467164    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:21.469865 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:21.469831    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:21.470471 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.470451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx" event={"ID":"1656b79b-ae4e-492c-b6a2-8af32ef633df","Type":"ContainerStarted","Data":"5c0361afc563c8cce1f32413d5f9238d6e481a2d0822f1b4640c327971e0b8d5"}
Jul 27 22:02:21.471863 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.471842    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7" event={"ID":"b38c447c-a4a5-4ba8-a324-3b8582e4d29c","Type":"ContainerStarted","Data":"fb2517f69b42490afd12130f92ed949b93062887b7fa8996a5ccad0426c0e246"}
Jul 27 22:02:21.472026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.471880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7" event={"ID":"b38c447c-a4a5-4ba8-a324-3b8582e4d29c","Type":"ContainerStarted","Data":"3c6a3301cf4ee1e59df2787e009718cb05f8b8bf5aa983034657deaca87197a4"}
Jul 27 22:02:21.472026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.471966    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:21.472223 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:21.472200    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:21.496026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.495980    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" podStartSLOduration=13.495965763 podStartE2EDuration="13.495965763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:21.493878131 +0000 UTC m=+451.680718792" watchObservedRunningTime="2026-07-27 22:02:21.495965763 +0000 UTC m=+451.682806422"
Jul 27 22:02:21.496244 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:21.496222    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dmq2v\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dmq2v"
Jul 27 22:02:21.519004 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.518958    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7" podStartSLOduration=13.518947334 podStartE2EDuration="13.518947334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:21.517765053 +0000 UTC m=+451.704605712" watchObservedRunningTime="2026-07-27 22:02:21.518947334 +0000 UTC m=+451.705788060"
Jul 27 22:02:21.519998 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:21.519975    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldmb7\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldmb7"
Jul 27 22:02:21.539895 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:21.539854    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j5qjx" podStartSLOduration=2.484649444 podStartE2EDuration="24.539841776s" totalImagesPullingTime="22.055192332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:21.53768974 +0000 UTC m=+451.724530400" watchObservedRunningTime="2026-07-27 22:02:21.539841776 +0000 UTC m=+451.726682477"
Jul 27 22:02:22.322865 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:22.322834    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6f0056-435e-4b74-9721-bdcbe1d3fea4" path="/var/lib/kubelet/pods/3f6f0056-435e-4b74-9721-bdcbe1d3fea4/volumes"
Jul 27 22:02:22.323220 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:22.323206    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca5aea8c-8499-4c14-be06-598466c0a58f" path="/var/lib/kubelet/pods/ca5aea8c-8499-4c14-be06-598466c0a58f/volumes"
Jul 27 22:02:22.844566 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:22.844527    2578 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-54q86_kuadrant-system_c6adfab2-c82c-413c-a1a9-2db40e8138ce_0(9902994b4c34d02b669ab8c211ed27c908f7323c317ab78722d2b03eaa644446): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-54q86 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 27 22:02:22.844848 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:22.844592    2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-54q86_kuadrant-system_c6adfab2-c82c-413c-a1a9-2db40e8138ce_0(9902994b4c34d02b669ab8c211ed27c908f7323c317ab78722d2b03eaa644446): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-54q86 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:23.481231 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.481203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:23.485097 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:23.485054    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-54q86\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="c6adfab2-c82c-413c-a1a9-2db40e8138ce" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:23.492181 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.492159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:23.495769 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:23.495743    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-54q86\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="c6adfab2-c82c-413c-a1a9-2db40e8138ce" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:23.588490 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.588461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvkdp\" (UniqueName: \"kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp\") pod \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") "
Jul 27 22:02:23.588623 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.588561    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume\") pod \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\" (UID: \"c6adfab2-c82c-413c-a1a9-2db40e8138ce\") "
Jul 27 22:02:23.588833 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.588813    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c6adfab2-c82c-413c-a1a9-2db40e8138ce" (UID: "c6adfab2-c82c-413c-a1a9-2db40e8138ce"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:23.590681 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.590650    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp" (OuterVolumeSpecName: "kube-api-access-pvkdp") pod "c6adfab2-c82c-413c-a1a9-2db40e8138ce" (UID: "c6adfab2-c82c-413c-a1a9-2db40e8138ce"). InnerVolumeSpecName "kube-api-access-pvkdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:23.689453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.689419    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6adfab2-c82c-413c-a1a9-2db40e8138ce-extensions-socket-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:23.689453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:23.689446    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvkdp\" (UniqueName: \"kubernetes.io/projected/c6adfab2-c82c-413c-a1a9-2db40e8138ce-kube-api-access-pvkdp\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:24.323111 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:24.323082    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6adfab2-c82c-413c-a1a9-2db40e8138ce" path="/var/lib/kubelet/pods/c6adfab2-c82c-413c-a1a9-2db40e8138ce/volumes"
Jul 27 22:02:24.485743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:24.485706    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:24.488592 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:24.488567    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-54q86\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="c6adfab2-c82c-413c-a1a9-2db40e8138ce" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:24.490889 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:24.490869    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-54q86\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="c6adfab2-c82c-413c-a1a9-2db40e8138ce" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-54q86"
Jul 27 22:02:31.505512 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.505465    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b6bdbfcd-8lfrc" podUID="fbf081ec-f378-4c7c-854e-c7976f527508" containerName="console" containerID="cri-o://2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc" gracePeriod=15
Jul 27 22:02:31.752622 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.752597    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 22:02:31.863785 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863698    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.863785 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863743    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.863785 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.864052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863794    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.864052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863830    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.864052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863850    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62pk6\" (UniqueName: \"kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.864052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.863895    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle\") pod \"fbf081ec-f378-4c7c-854e-c7976f527508\" (UID: \"fbf081ec-f378-4c7c-854e-c7976f527508\") "
Jul 27 22:02:31.864259 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.864115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config" (OuterVolumeSpecName: "console-config") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:31.864259 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.864226    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca" (OuterVolumeSpecName: "service-ca") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:31.864367 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.864233    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:31.864476 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.864456    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:02:31.866176 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.866153    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:02:31.866281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.866191    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:02:31.866281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.866213    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6" (OuterVolumeSpecName: "kube-api-access-62pk6") pod "fbf081ec-f378-4c7c-854e-c7976f527508" (UID: "fbf081ec-f378-4c7c-854e-c7976f527508"). InnerVolumeSpecName "kube-api-access-62pk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:31.964532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964495    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964526    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-oauth-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964536    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-oauth-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964545    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62pk6\" (UniqueName: \"kubernetes.io/projected/fbf081ec-f378-4c7c-854e-c7976f527508-kube-api-access-62pk6\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964555    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-trusted-ca-bundle\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964563    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf081ec-f378-4c7c-854e-c7976f527508-console-config\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:31.964799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:31.964571    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf081ec-f378-4c7c-854e-c7976f527508-console-serving-cert\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:32.478843 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.478811    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6ntp7"
Jul 27 22:02:32.479014 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.478996    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:32.527158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.527127    2578 generic.go:356] "Generic (PLEG): container finished" podID="fbf081ec-f378-4c7c-854e-c7976f527508" containerID="2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc" exitCode=0
Jul 27 22:02:32.527587 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.527176    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6bdbfcd-8lfrc"
Jul 27 22:02:32.527587 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.527202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6bdbfcd-8lfrc" event={"ID":"fbf081ec-f378-4c7c-854e-c7976f527508","Type":"ContainerDied","Data":"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"}
Jul 27 22:02:32.527587 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.527250    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6bdbfcd-8lfrc" event={"ID":"fbf081ec-f378-4c7c-854e-c7976f527508","Type":"ContainerDied","Data":"aee7ea14b444d3ca8142aeb850541c8d513754df232f4dcaac6e45fe16eb63e9"}
Jul 27 22:02:32.527587 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.527272    2578 scope.go:117] "RemoveContainer" containerID="2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"
Jul 27 22:02:32.536610 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.536588    2578 scope.go:117] "RemoveContainer" containerID="2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"
Jul 27 22:02:32.536910 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:32.536882    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc\": container with ID starting with 2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc not found: ID does not exist" containerID="2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"
Jul 27 22:02:32.536982 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.536915    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc"} err="failed to get container status \"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc\": rpc error: code = NotFound desc = could not find container \"2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc\": container with ID starting with 2dffd75fb5068cc2e908f0d1a5aa90d5d612bc859302f138b0a0ce302f9f35cc not found: ID does not exist"
Jul 27 22:02:32.562541 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.562516    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 22:02:32.566712 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.566688    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b6bdbfcd-8lfrc"]
Jul 27 22:02:32.930983 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.930951    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:02:32.931304 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.931292    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf081ec-f378-4c7c-854e-c7976f527508" containerName="console"
Jul 27 22:02:32.931346 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.931306    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf081ec-f378-4c7c-854e-c7976f527508" containerName="console"
Jul 27 22:02:32.931387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.931378    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf081ec-f378-4c7c-854e-c7976f527508" containerName="console"
Jul 27 22:02:32.933959 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.933944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:32.949827 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:32.949798    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:02:33.074281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.074245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.074453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.074345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8qp2\" (UniqueName: \"kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.175495 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.175408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8qp2\" (UniqueName: \"kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.175495 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.175466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.175827 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.175810    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.186937 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.186913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8qp2\" (UniqueName: \"kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2\") pod \"kuadrant-operator-controller-manager-86878c769f-fhxxz\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.243476 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.243441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.384040 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.384016    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:02:33.386529 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:02:33.386495    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7393cd4e_db8c_47c4_937c_bfe4aedb31ef.slice/crio-7814eea3acf869539c0f7b97a54e47caa815103b47ae38f73c5c06a775cc5500 WatchSource:0}: Error finding container 7814eea3acf869539c0f7b97a54e47caa815103b47ae38f73c5c06a775cc5500: Status 404 returned error can't find the container with id 7814eea3acf869539c0f7b97a54e47caa815103b47ae38f73c5c06a775cc5500
Jul 27 22:02:33.534948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.534914    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" event={"ID":"7393cd4e-db8c-47c4-937c-bfe4aedb31ef","Type":"ContainerStarted","Data":"b4b6a5907338ed77d3fe414845fcb4f9cd080989eaedb3c91aae547f30384e50"}
Jul 27 22:02:33.535340 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.534972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" event={"ID":"7393cd4e-db8c-47c4-937c-bfe4aedb31ef","Type":"ContainerStarted","Data":"7814eea3acf869539c0f7b97a54e47caa815103b47ae38f73c5c06a775cc5500"}
Jul 27 22:02:33.535340 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.535049    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:33.557496 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:33.557447    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" podStartSLOduration=1.557433243 podStartE2EDuration="1.557433243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:33.555984442 +0000 UTC m=+463.742825102" watchObservedRunningTime="2026-07-27 22:02:33.557433243 +0000 UTC m=+463.744273902"
Jul 27 22:02:34.323170 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:34.323135    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf081ec-f378-4c7c-854e-c7976f527508" path="/var/lib/kubelet/pods/fbf081ec-f378-4c7c-854e-c7976f527508/volumes"
Jul 27 22:02:44.543018 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.542987    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:02:44.597131 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.597104    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:44.597302 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.597279    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" podUID="e9c530dc-ad29-45e1-9b22-95bf59be455c" containerName="manager" containerID="cri-o://280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c" gracePeriod=10
Jul 27 22:02:44.875241 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.875213    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:44.982798 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.982765    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume\") pod \"e9c530dc-ad29-45e1-9b22-95bf59be455c\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") "
Jul 27 22:02:44.982959 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.982825    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlz29\" (UniqueName: \"kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29\") pod \"e9c530dc-ad29-45e1-9b22-95bf59be455c\" (UID: \"e9c530dc-ad29-45e1-9b22-95bf59be455c\") "
Jul 27 22:02:44.983125 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.983104    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e9c530dc-ad29-45e1-9b22-95bf59be455c" (UID: "e9c530dc-ad29-45e1-9b22-95bf59be455c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:02:44.984978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:44.984945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29" (OuterVolumeSpecName: "kube-api-access-mlz29") pod "e9c530dc-ad29-45e1-9b22-95bf59be455c" (UID: "e9c530dc-ad29-45e1-9b22-95bf59be455c"). InnerVolumeSpecName "kube-api-access-mlz29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:45.083680 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.083645    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlz29\" (UniqueName: \"kubernetes.io/projected/e9c530dc-ad29-45e1-9b22-95bf59be455c-kube-api-access-mlz29\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:45.083843 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.083695    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e9c530dc-ad29-45e1-9b22-95bf59be455c-extensions-socket-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:45.598175 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.598140    2578 generic.go:356] "Generic (PLEG): container finished" podID="e9c530dc-ad29-45e1-9b22-95bf59be455c" containerID="280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c" exitCode=0
Jul 27 22:02:45.598625 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.598178    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"
Jul 27 22:02:45.598625 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.598224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" event={"ID":"e9c530dc-ad29-45e1-9b22-95bf59be455c","Type":"ContainerDied","Data":"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"}
Jul 27 22:02:45.598625 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.598282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg" event={"ID":"e9c530dc-ad29-45e1-9b22-95bf59be455c","Type":"ContainerDied","Data":"3dcf7f6d19c2542c6f5badf50b89b087e9e5246633ebb68198cef22c677cdb0b"}
Jul 27 22:02:45.598625 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.598314    2578 scope.go:117] "RemoveContainer" containerID="280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"
Jul 27 22:02:45.607192 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.607176    2578 scope.go:117] "RemoveContainer" containerID="280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"
Jul 27 22:02:45.607444 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:45.607426    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c\": container with ID starting with 280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c not found: ID does not exist" containerID="280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"
Jul 27 22:02:45.607491 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.607452    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c"} err="failed to get container status \"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c\": rpc error: code = NotFound desc = could not find container \"280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c\": container with ID starting with 280e70625de617648000443cb0a5724f5dd3baf87b652debee81718ca066703c not found: ID does not exist"
Jul 27 22:02:45.627836 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.627776    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:45.630060 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:45.630041    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-svkbg"]
Jul 27 22:02:46.322739 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:46.322705    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c530dc-ad29-45e1-9b22-95bf59be455c" path="/var/lib/kubelet/pods/e9c530dc-ad29-45e1-9b22-95bf59be455c/volumes"
Jul 27 22:02:52.767889 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.767857    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:02:52.768276 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.768265    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c530dc-ad29-45e1-9b22-95bf59be455c" containerName="manager"
Jul 27 22:02:52.768322 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.768278    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c530dc-ad29-45e1-9b22-95bf59be455c" containerName="manager"
Jul 27 22:02:52.768354 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.768345    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c530dc-ad29-45e1-9b22-95bf59be455c" containerName="manager"
Jul 27 22:02:52.789394 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.789369    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:02:52.789538 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.789474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:52.792600 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.792579    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 22:02:52.794198 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.794173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:02:52.951069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.951034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2q7\" (UniqueName: \"kubernetes.io/projected/8dab191a-5547-44e2-a9d4-687d831ee5e7-kube-api-access-9h2q7\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:52.951232 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:52.951101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8dab191a-5547-44e2-a9d4-687d831ee5e7-config-file\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.051964 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.051881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2q7\" (UniqueName: \"kubernetes.io/projected/8dab191a-5547-44e2-a9d4-687d831ee5e7-kube-api-access-9h2q7\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.051964 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.051952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8dab191a-5547-44e2-a9d4-687d831ee5e7-config-file\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.052525 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.052505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8dab191a-5547-44e2-a9d4-687d831ee5e7-config-file\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.061680 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.061649    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2q7\" (UniqueName: \"kubernetes.io/projected/8dab191a-5547-44e2-a9d4-687d831ee5e7-kube-api-access-9h2q7\") pod \"limitador-limitador-54d6d988d5-86dbt\" (UID: \"8dab191a-5547-44e2-a9d4-687d831ee5e7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.100371 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.100347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:53.120034 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.120007    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:02:53.123212 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.122539    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:53.125595 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.125566    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7r4kf\""
Jul 27 22:02:53.129663 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.129640    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:02:53.152374 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.152348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh745\" (UniqueName: \"kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745\") pod \"authorino-58cd955675-mwnf9\" (UID: \"25c0fed3-dda8-4c1f-b275-7a9bc63342e9\") " pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:53.244138 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.243999    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:02:53.246393 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:02:53.246367    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dab191a_5547_44e2_a9d4_687d831ee5e7.slice/crio-09441110537fc7c7027c66adba46cd9f5c5a0b653e1de284f7e6d3c55e88651c WatchSource:0}: Error finding container 09441110537fc7c7027c66adba46cd9f5c5a0b653e1de284f7e6d3c55e88651c: Status 404 returned error can't find the container with id 09441110537fc7c7027c66adba46cd9f5c5a0b653e1de284f7e6d3c55e88651c
Jul 27 22:02:53.253541 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.253393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh745\" (UniqueName: \"kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745\") pod \"authorino-58cd955675-mwnf9\" (UID: \"25c0fed3-dda8-4c1f-b275-7a9bc63342e9\") " pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:53.266434 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.266408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh745\" (UniqueName: \"kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745\") pod \"authorino-58cd955675-mwnf9\" (UID: \"25c0fed3-dda8-4c1f-b275-7a9bc63342e9\") " pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:53.434318 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.434237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:53.581867 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.581837    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:02:53.584659 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:02:53.584621    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25c0fed3_dda8_4c1f_b275_7a9bc63342e9.slice/crio-daf1af59b31b27f2c2a0a1961a88ff7481436039fbea52102f90349f04629247 WatchSource:0}: Error finding container daf1af59b31b27f2c2a0a1961a88ff7481436039fbea52102f90349f04629247: Status 404 returned error can't find the container with id daf1af59b31b27f2c2a0a1961a88ff7481436039fbea52102f90349f04629247
Jul 27 22:02:53.638645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.638617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwnf9" event={"ID":"25c0fed3-dda8-4c1f-b275-7a9bc63342e9","Type":"ContainerStarted","Data":"daf1af59b31b27f2c2a0a1961a88ff7481436039fbea52102f90349f04629247"}
Jul 27 22:02:53.639809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:53.639784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt" event={"ID":"8dab191a-5547-44e2-a9d4-687d831ee5e7","Type":"ContainerStarted","Data":"09441110537fc7c7027c66adba46cd9f5c5a0b653e1de284f7e6d3c55e88651c"}
Jul 27 22:02:57.141559 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:57.141525    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:02:58.669492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.669453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwnf9" event={"ID":"25c0fed3-dda8-4c1f-b275-7a9bc63342e9","Type":"ContainerStarted","Data":"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"}
Jul 27 22:02:58.669973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.669513    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mwnf9" podUID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" containerName="authorino" containerID="cri-o://ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5" gracePeriod=30
Jul 27 22:02:58.671076 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.671052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt" event={"ID":"8dab191a-5547-44e2-a9d4-687d831ee5e7","Type":"ContainerStarted","Data":"9c4029904243a957f3fad8e92675dee17e1394f86b0aa81ca7271327978e364e"}
Jul 27 22:02:58.671239 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.671226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:02:58.685278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.685235    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mwnf9" podStartSLOduration=1.471455888 podStartE2EDuration="5.685217519s" totalImagesPullingTime="4.213761631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:58.684254146 +0000 UTC m=+488.871094806" watchObservedRunningTime="2026-07-27 22:02:58.685217519 +0000 UTC m=+488.872058156"
Jul 27 22:02:58.700752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.700704    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt" podStartSLOduration=2.111668733 podStartE2EDuration="6.700659265s" totalImagesPullingTime="4.588990532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:02:58.699886816 +0000 UTC m=+488.886727479" watchObservedRunningTime="2026-07-27 22:02:58.700659265 +0000 UTC m=+488.887499923"
Jul 27 22:02:58.920019 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.919957    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:58.999505 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:58.999475    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh745\" (UniqueName: \"kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745\") pod \"25c0fed3-dda8-4c1f-b275-7a9bc63342e9\" (UID: \"25c0fed3-dda8-4c1f-b275-7a9bc63342e9\") "
Jul 27 22:02:59.001743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.001709    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745" (OuterVolumeSpecName: "kube-api-access-xh745") pod "25c0fed3-dda8-4c1f-b275-7a9bc63342e9" (UID: "25c0fed3-dda8-4c1f-b275-7a9bc63342e9"). InnerVolumeSpecName "kube-api-access-xh745". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:02:59.100509 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.100465    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh745\" (UniqueName: \"kubernetes.io/projected/25c0fed3-dda8-4c1f-b275-7a9bc63342e9-kube-api-access-xh745\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:02:59.676774 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.676742    2578 generic.go:356] "Generic (PLEG): container finished" podID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" containerID="ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5" exitCode=0
Jul 27 22:02:59.677209 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.676785    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwnf9"
Jul 27 22:02:59.677209 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.676835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwnf9" event={"ID":"25c0fed3-dda8-4c1f-b275-7a9bc63342e9","Type":"ContainerDied","Data":"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"}
Jul 27 22:02:59.677209 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.676877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwnf9" event={"ID":"25c0fed3-dda8-4c1f-b275-7a9bc63342e9","Type":"ContainerDied","Data":"daf1af59b31b27f2c2a0a1961a88ff7481436039fbea52102f90349f04629247"}
Jul 27 22:02:59.677209 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.676898    2578 scope.go:117] "RemoveContainer" containerID="ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"
Jul 27 22:02:59.685832 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.685816    2578 scope.go:117] "RemoveContainer" containerID="ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"
Jul 27 22:02:59.686085 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:02:59.686063    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5\": container with ID starting with ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5 not found: ID does not exist" containerID="ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"
Jul 27 22:02:59.686160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.686097    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5"} err="failed to get container status \"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5\": rpc error: code = NotFound desc = could not find container \"ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5\": container with ID starting with ebc4e494aad535350a56d7136fd9ee38a5df2fc9279b9a93cb2ffe52652d73b5 not found: ID does not exist"
Jul 27 22:02:59.697134 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.697108    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:02:59.701140 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:02:59.701118    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwnf9"]
Jul 27 22:03:00.322299 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:03:00.322268    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" path="/var/lib/kubelet/pods/25c0fed3-dda8-4c1f-b275-7a9bc63342e9/volumes"
Jul 27 22:03:09.678276 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:03:09.678244    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-86dbt"
Jul 27 22:04:08.557624 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:08.557586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:04:44.144899 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:44.144859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:04:50.176603 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:50.176577    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:04:50.177537 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:50.177518    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:04:54.139784 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.139751    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"]
Jul 27 22:04:54.140143 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.140127    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" containerName="authorino"
Jul 27 22:04:54.140143 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.140138    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" containerName="authorino"
Jul 27 22:04:54.140256 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.140204    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c0fed3-dda8-4c1f-b275-7a9bc63342e9" containerName="authorino"
Jul 27 22:04:54.143353 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.143333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.146324 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.146302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mjxw8\""
Jul 27 22:04:54.146324 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.146318    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 22:04:54.147204 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.147183    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 22:04:54.147302 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.147239    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 22:04:54.159027 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.158998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"]
Jul 27 22:04:54.272025 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.271997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.272025 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.272030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndh7l\" (UniqueName: \"kubernetes.io/projected/d7a2bd50-5898-42ef-8dc8-825d3daad16d-kube-api-access-ndh7l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.272245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.272066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.272245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.272118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.272245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.272151    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.272245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.272171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373446 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndh7l\" (UniqueName: \"kubernetes.io/projected/d7a2bd50-5898-42ef-8dc8-825d3daad16d-kube-api-access-ndh7l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.373995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.373984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.374083 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.374050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.375935 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.375915    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7a2bd50-5898-42ef-8dc8-825d3daad16d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.376237 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.376217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7a2bd50-5898-42ef-8dc8-825d3daad16d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.383958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.383912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndh7l\" (UniqueName: \"kubernetes.io/projected/d7a2bd50-5898-42ef-8dc8-825d3daad16d-kube-api-access-ndh7l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-72lvt\" (UID: \"d7a2bd50-5898-42ef-8dc8-825d3daad16d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.453250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.453171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:04:54.454195 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.454173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:04:54.595597 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:54.595569    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"]
Jul 27 22:04:54.598405 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:04:54.598374    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a2bd50_5898_42ef_8dc8_825d3daad16d.slice/crio-3423a6884865eabbbfdc426259985347ea276f901f5304df5ae1a62fab880bb2 WatchSource:0}: Error finding container 3423a6884865eabbbfdc426259985347ea276f901f5304df5ae1a62fab880bb2: Status 404 returned error can't find the container with id 3423a6884865eabbbfdc426259985347ea276f901f5304df5ae1a62fab880bb2
Jul 27 22:04:55.245955 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:55.245921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt" event={"ID":"d7a2bd50-5898-42ef-8dc8-825d3daad16d","Type":"ContainerStarted","Data":"3423a6884865eabbbfdc426259985347ea276f901f5304df5ae1a62fab880bb2"}
Jul 27 22:04:56.252222 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:04:56.252187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt" event={"ID":"d7a2bd50-5898-42ef-8dc8-825d3daad16d","Type":"ContainerStarted","Data":"e3a4c7cf336b2729dcbcd8cf60aff8444dfb425e808b0a5918856937d8192aaf"}
Jul 27 22:05:04.453944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:04.453906    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:05:04.454403 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:04.453975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:05:04.467141 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:04.467112    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:05:04.487567 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:04.487517    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt" podStartSLOduration=9.51694007 podStartE2EDuration="10.487500999s" totalImagesPullingTime="970.560929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:04:56.297405714 +0000 UTC m=+606.484246373" watchObservedRunningTime="2026-07-27 22:05:04.487500999 +0000 UTC m=+614.674341652"
Jul 27 22:05:05.310232 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:05.310199    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-72lvt"
Jul 27 22:05:08.249985 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:08.249952    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:05:21.693194 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:21.693113    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:05:26.538716 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.538683    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"]
Jul 27 22:05:26.542387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.542365    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.545052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.545023    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 22:05:26.553857 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.553834    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"]
Jul 27 22:05:26.633060 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.633060 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.633278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.633278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9c4f\" (UniqueName: \"kubernetes.io/projected/a67bbc9d-22e2-48b2-9275-44d3171921f1-kube-api-access-x9c4f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.633278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a67bbc9d-22e2-48b2-9275-44d3171921f1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.633386 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.633278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a67bbc9d-22e2-48b2-9275-44d3171921f1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734300 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734300 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734300 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734300 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734485 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9c4f\" (UniqueName: \"kubernetes.io/projected/a67bbc9d-22e2-48b2-9275-44d3171921f1-kube-api-access-x9c4f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734654 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734787 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.734787 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.734714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.736709 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.736655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a67bbc9d-22e2-48b2-9275-44d3171921f1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.736802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.736774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a67bbc9d-22e2-48b2-9275-44d3171921f1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.744273 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.744249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9c4f\" (UniqueName: \"kubernetes.io/projected/a67bbc9d-22e2-48b2-9275-44d3171921f1-kube-api-access-x9c4f\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4vlx9\" (UID: \"a67bbc9d-22e2-48b2-9275-44d3171921f1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.853060 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.852968    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:26.997108 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:26.997071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"]
Jul 27 22:05:27.004470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:27.004435    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:05:27.408519 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:27.408479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9" event={"ID":"a67bbc9d-22e2-48b2-9275-44d3171921f1","Type":"ContainerStarted","Data":"673a1058bdc3d14fe3cd006b9fd249805ff1bf96d4c5e5a71359b15da96db27c"}
Jul 27 22:05:27.408694 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:27.408550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9" event={"ID":"a67bbc9d-22e2-48b2-9275-44d3171921f1","Type":"ContainerStarted","Data":"2e8844c4be6c2bd3b8e42aec498b492d6ffcdb13e2d39236aa491c815ed91058"}
Jul 27 22:05:27.429415 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:27.429364    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9" podStartSLOduration=1.2233867489999999 podStartE2EDuration="1.429349184s" totalImagesPullingTime="205.962435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:05:27.42808194 +0000 UTC m=+637.614922599" watchObservedRunningTime="2026-07-27 22:05:27.429349184 +0000 UTC m=+637.616189856"
Jul 27 22:05:27.950492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:27.950462    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:05:36.853729 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:36.853691    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:36.854133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:36.853750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:36.866150 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:36.866126    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:37.470660 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:37.470632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4vlx9"
Jul 27 22:05:47.558847 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:05:47.558815    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:07:07.251307 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:07:07.251215    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:07:20.544249 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:07:20.544217    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:07:28.692443 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:07:28.692406    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:09:09.157361 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:09.157324    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:09:17.263639 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:17.263601    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:09:47.850440 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:47.850358    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:09:50.208902 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:50.208871    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:09:50.209589 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:50.209558    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:09:55.968700 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:09:55.968648    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:06.655503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:06.655464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:13.948417 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:13.948380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:23.441578 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:23.441539    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:34.777086 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:34.777050    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:45.358905 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:45.358869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:10:54.959265 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:10:54.959234    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:11:16.561762 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:11:16.561659    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:11:25.891291 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:11:25.891247    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:11:50.859316 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:11:50.859281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:02.660945 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:02.660905    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:18.156088 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:18.156052    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:27.150363 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:27.150317    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:36.952820 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:36.952782    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:46.159523 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:46.159438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:12:54.658143 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:12:54.658113    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:13:23.847441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:23.847405    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:13:23.847993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:23.847596    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" podUID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" containerName="manager" containerID="cri-o://b4b6a5907338ed77d3fe414845fcb4f9cd080989eaedb3c91aae547f30384e50" gracePeriod=10
Jul 27 22:13:27.742472 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:27.742439    2578 generic.go:356] "Generic (PLEG): container finished" podID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" containerID="b4b6a5907338ed77d3fe414845fcb4f9cd080989eaedb3c91aae547f30384e50" exitCode=0
Jul 27 22:13:27.742871 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:27.742508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" event={"ID":"7393cd4e-db8c-47c4-937c-bfe4aedb31ef","Type":"ContainerDied","Data":"b4b6a5907338ed77d3fe414845fcb4f9cd080989eaedb3c91aae547f30384e50"}
Jul 27 22:13:28.104348 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.104321    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:13:28.168949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.168918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume\") pod \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") "
Jul 27 22:13:28.168949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.168955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8qp2\" (UniqueName: \"kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2\") pod \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\" (UID: \"7393cd4e-db8c-47c4-937c-bfe4aedb31ef\") "
Jul 27 22:13:28.169343 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.169316    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7393cd4e-db8c-47c4-937c-bfe4aedb31ef" (UID: "7393cd4e-db8c-47c4-937c-bfe4aedb31ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:13:28.171150 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.171130    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2" (OuterVolumeSpecName: "kube-api-access-z8qp2") pod "7393cd4e-db8c-47c4-937c-bfe4aedb31ef" (UID: "7393cd4e-db8c-47c4-937c-bfe4aedb31ef"). InnerVolumeSpecName "kube-api-access-z8qp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:13:28.269991 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.269914    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-extensions-socket-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:13:28.269991 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.269939    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8qp2\" (UniqueName: \"kubernetes.io/projected/7393cd4e-db8c-47c4-937c-bfe4aedb31ef-kube-api-access-z8qp2\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:13:28.755310 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.755277    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz" event={"ID":"7393cd4e-db8c-47c4-937c-bfe4aedb31ef","Type":"ContainerDied","Data":"7814eea3acf869539c0f7b97a54e47caa815103b47ae38f73c5c06a775cc5500"}
Jul 27 22:13:28.755310 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.755295    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"
Jul 27 22:13:28.755796 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.755361    2578 scope.go:117] "RemoveContainer" containerID="b4b6a5907338ed77d3fe414845fcb4f9cd080989eaedb3c91aae547f30384e50"
Jul 27 22:13:28.776282 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.776251    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:13:28.786158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:28.786135    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fhxxz"]
Jul 27 22:13:30.322366 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:13:30.322322    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" path="/var/lib/kubelet/pods/7393cd4e-db8c-47c4-937c-bfe4aedb31ef/volumes"
Jul 27 22:14:24.968074 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.967984    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"]
Jul 27 22:14:24.968641 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.968533    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" containerName="manager"
Jul 27 22:14:24.968641 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.968554    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" containerName="manager"
Jul 27 22:14:24.968802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.968649    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7393cd4e-db8c-47c4-937c-bfe4aedb31ef" containerName="manager"
Jul 27 22:14:24.971711 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.971659    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:24.974917 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.974899    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gb9r2\""
Jul 27 22:14:24.988188 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:24.988162    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"]
Jul 27 22:14:25.060262 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.060227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92ce823-3263-4c5f-8b84-71c33ce922a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.060444 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.060291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4kcs\" (UniqueName: \"kubernetes.io/projected/b92ce823-3263-4c5f-8b84-71c33ce922a2-kube-api-access-j4kcs\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.160944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.160908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92ce823-3263-4c5f-8b84-71c33ce922a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.161105 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.160959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4kcs\" (UniqueName: \"kubernetes.io/projected/b92ce823-3263-4c5f-8b84-71c33ce922a2-kube-api-access-j4kcs\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.161306 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.161285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92ce823-3263-4c5f-8b84-71c33ce922a2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.174305 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.174275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4kcs\" (UniqueName: \"kubernetes.io/projected/b92ce823-3263-4c5f-8b84-71c33ce922a2-kube-api-access-j4kcs\") pod \"kuadrant-operator-controller-manager-86878c769f-vc2kg\" (UID: \"b92ce823-3263-4c5f-8b84-71c33ce922a2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.282022 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.281977    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:25.436914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:25.436891    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"]
Jul 27 22:14:25.439038 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:14:25.439012    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92ce823_3263_4c5f_8b84_71c33ce922a2.slice/crio-7fe7c27294054302aa764358394ac32e2e6eb83e4e7e0beab037148cbe1ffb5c WatchSource:0}: Error finding container 7fe7c27294054302aa764358394ac32e2e6eb83e4e7e0beab037148cbe1ffb5c: Status 404 returned error can't find the container with id 7fe7c27294054302aa764358394ac32e2e6eb83e4e7e0beab037148cbe1ffb5c
Jul 27 22:14:26.035432 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:26.035399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg" event={"ID":"b92ce823-3263-4c5f-8b84-71c33ce922a2","Type":"ContainerStarted","Data":"f23a692ae5d2d8a2792682306cd26c7777ff53fd95de6d07c5e5dfd41bf543e8"}
Jul 27 22:14:26.035858 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:26.035455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg" event={"ID":"b92ce823-3263-4c5f-8b84-71c33ce922a2","Type":"ContainerStarted","Data":"7fe7c27294054302aa764358394ac32e2e6eb83e4e7e0beab037148cbe1ffb5c"}
Jul 27 22:14:26.035858 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:26.035529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:26.069837 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:26.069662    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg" podStartSLOduration=2.069642212 podStartE2EDuration="2.069642212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:14:26.067811065 +0000 UTC m=+1176.254651735" watchObservedRunningTime="2026-07-27 22:14:26.069642212 +0000 UTC m=+1176.256482873"
Jul 27 22:14:37.042471 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:37.042442    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vc2kg"
Jul 27 22:14:46.552226 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:46.552187    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:14:50.236123 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:50.236092    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:14:50.238437 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:50.238413    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:14:58.358045 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:14:58.358012    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:15:07.359909 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:15:07.359872    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:15:19.456460 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:15:19.456416    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:15:31.067618 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:15:31.067578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:15:40.453819 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:15:40.453781    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:15:49.838539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:15:49.838452    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:16:00.772886 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:16:00.772852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:16:12.758014 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:16:12.757974    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:16:21.146327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:16:21.146284    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:16:32.490080 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:16:32.490042    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:16:53.118589 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:16:53.118557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:17:03.576107 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:03.576069    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:17:21.970419 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:21.970303    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:17:21.974059 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:21.974037    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:21.976990 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:21.976968    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cc5625f3-kserve-self-signed-certs\""
Jul 27 22:17:21.995377 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:21.995344    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:17:22.019845 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.019978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.019978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.019978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.019978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.020154 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.019996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44fmc\" (UniqueName: \"kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.120906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.120875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.120906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.120913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121181 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.120936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44fmc\" (UniqueName: \"kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121181 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.120975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121181 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.121005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121181 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.121027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121405 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.121380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.121414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.121493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.121459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.123448 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.123418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.123718 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.123704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.138810 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.138785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44fmc\" (UniqueName: \"kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc\") pod \"e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") " pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.283480 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.283443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:22.427660 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.427638    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:17:22.429642 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:17:22.429601    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode31a351c_5a74_4061_bc42_41fdeb338284.slice/crio-12b35a6638780d2593ba0e349a1b302f7c84d29f62ce8f50b72fe3cdade1e994 WatchSource:0}: Error finding container 12b35a6638780d2593ba0e349a1b302f7c84d29f62ce8f50b72fe3cdade1e994: Status 404 returned error can't find the container with id 12b35a6638780d2593ba0e349a1b302f7c84d29f62ce8f50b72fe3cdade1e994
Jul 27 22:17:22.899860 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.899826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" event={"ID":"e31a351c-5a74-4061-bc42-41fdeb338284","Type":"ContainerStarted","Data":"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"}
Jul 27 22:17:22.899860 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.899873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" event={"ID":"e31a351c-5a74-4061-bc42-41fdeb338284","Type":"ContainerStarted","Data":"12b35a6638780d2593ba0e349a1b302f7c84d29f62ce8f50b72fe3cdade1e994"}
Jul 27 22:17:22.923248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:22.923202    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" podStartSLOduration=1.923186965 podStartE2EDuration="1.923186965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:17:22.919655202 +0000 UTC m=+1353.106495864" watchObservedRunningTime="2026-07-27 22:17:22.923186965 +0000 UTC m=+1353.110027623"
Jul 27 22:17:24.862803 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:24.862762    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:17:32.283863 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:32.283761    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:32.284312 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:32.284072    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:32.296731 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:32.296706    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:32.961092 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:32.961054    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:17:41.649216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:17:41.649180    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:18:13.726472 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:13.726438    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:18:17.363208 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:17.363171    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:18:28.746217 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:28.746175    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" podUID="e31a351c-5a74-4061-bc42-41fdeb338284" containerName="main" containerID="cri-o://dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65" gracePeriod=45
Jul 27 22:18:29.005545 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.005487    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:18:29.121289 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121311    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121413    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121465    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44fmc\" (UniqueName: \"kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc\") pod \"e31a351c-5a74-4061-bc42-41fdeb338284\" (UID: \"e31a351c-5a74-4061-bc42-41fdeb338284\") "
Jul 27 22:18:29.121650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121580    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home" (OuterVolumeSpecName: "home") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:18:29.121875 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.121850    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.122042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.122017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache" (OuterVolumeSpecName: "model-cache") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:18:29.122115 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.122086    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:18:29.123793 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.123764    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc" (OuterVolumeSpecName: "kube-api-access-44fmc") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "kube-api-access-44fmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:18:29.123890 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.123799    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:18:29.123890 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.123804    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm" (OuterVolumeSpecName: "dshm") pod "e31a351c-5a74-4061-bc42-41fdeb338284" (UID: "e31a351c-5a74-4061-bc42-41fdeb338284"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:18:29.222731 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.222696    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31a351c-5a74-4061-bc42-41fdeb338284-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.222731 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.222724    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.222970 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.222760    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.222970 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.222771    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31a351c-5a74-4061-bc42-41fdeb338284-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.222970 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.222783    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44fmc\" (UniqueName: \"kubernetes.io/projected/e31a351c-5a74-4061-bc42-41fdeb338284-kube-api-access-44fmc\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:18:29.228991 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.228962    2578 generic.go:356] "Generic (PLEG): container finished" podID="e31a351c-5a74-4061-bc42-41fdeb338284" containerID="dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65" exitCode=0
Jul 27 22:18:29.229121 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.229006    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"
Jul 27 22:18:29.229121 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.229048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" event={"ID":"e31a351c-5a74-4061-bc42-41fdeb338284","Type":"ContainerDied","Data":"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"}
Jul 27 22:18:29.229121 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.229093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m" event={"ID":"e31a351c-5a74-4061-bc42-41fdeb338284","Type":"ContainerDied","Data":"12b35a6638780d2593ba0e349a1b302f7c84d29f62ce8f50b72fe3cdade1e994"}
Jul 27 22:18:29.229121 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.229116    2578 scope.go:117] "RemoveContainer" containerID="dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"
Jul 27 22:18:29.238077 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.238062    2578 scope.go:117] "RemoveContainer" containerID="dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"
Jul 27 22:18:29.238331 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:18:29.238303    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65\": container with ID starting with dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65 not found: ID does not exist" containerID="dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"
Jul 27 22:18:29.238380 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.238329    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65"} err="failed to get container status \"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65\": rpc error: code = NotFound desc = could not find container \"dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65\": container with ID starting with dcb2b1be9b9d7dd9646bf31f4fb0b24f05a530ad412a5cdfef3ddafa61669a65 not found: ID does not exist"
Jul 27 22:18:29.249868 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.249843    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:18:29.254004 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:29.253984    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cc5625f3-kserve-74fdf685b7-9742m"]
Jul 27 22:18:30.322953 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:30.322911    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e31a351c-5a74-4061-bc42-41fdeb338284" path="/var/lib/kubelet/pods/e31a351c-5a74-4061-bc42-41fdeb338284/volumes"
Jul 27 22:18:31.640146 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:31.640108    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:18:38.347076 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:38.347041    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:18:47.259705 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:47.259594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:18:55.148967 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:18:55.148934    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:04.639524 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:04.639495    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:13.753567 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:13.753535    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:22.936794 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:22.936763    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:33.549568 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:33.549533    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:43.342980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:43.342951    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:19:50.265857 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:50.265813    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:19:50.269384 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:50.269360    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:19:56.549878 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:19:56.549843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:05.436947 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:05.436909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:14.746799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:14.746705    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:27.137168 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:27.137138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:37.453117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:37.453077    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:47.345493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:47.345455    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:20:59.148300 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:20:59.148264    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:11.048465 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:11.048429    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:19.947142 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:19.947100    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:33.941402 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:33.941369    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:43.640897 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:43.640814    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:53.445617 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:53.445573    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:21:56.829262 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.829228    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:21:56.829657 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.829590    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e31a351c-5a74-4061-bc42-41fdeb338284" containerName="main"
Jul 27 22:21:56.829657 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.829601    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e31a351c-5a74-4061-bc42-41fdeb338284" containerName="main"
Jul 27 22:21:56.829760 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.829687    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e31a351c-5a74-4061-bc42-41fdeb338284" containerName="main"
Jul 27 22:21:56.832904 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.832888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.836036 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.836015    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b8ed72f2\""
Jul 27 22:21:56.843087 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.843067    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:21:56.874349 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.874328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.874455 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.874394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w86jz\" (UniqueName: \"kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.877603 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.877583    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:21:56.881061 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.881048    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:56.883883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.883852    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b8ed72f2-dockercfg-7mb4d\""
Jul 27 22:21:56.889293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.889274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:21:56.974974 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.974944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86jz\" (UniqueName: \"kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.974974 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.974983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:56.975213 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.975036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.975213 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.975120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbk44\" (UniqueName: \"kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:56.975828 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.975804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:56.983980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:56.983958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w86jz\" (UniqueName: \"kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz\") pod \"payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:57.075823 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.075795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:57.075979 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.075886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbk44\" (UniqueName: \"kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:57.076518 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.076498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:57.084602 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.084540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbk44\" (UniqueName: \"kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44\") pod \"payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") " pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:57.143599 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.143562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:57.192658 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.192631    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:57.295391 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.295359    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:21:57.297422 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:21:57.297389    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64dc3969_dcc7_42d7_aaed_ef816b19715f.slice/crio-16f71df659e8b59c03b9a461a32e232e3dbf7d67d0781ebb5093e23bd08c86e4 WatchSource:0}: Error finding container 16f71df659e8b59c03b9a461a32e232e3dbf7d67d0781ebb5093e23bd08c86e4: Status 404 returned error can't find the container with id 16f71df659e8b59c03b9a461a32e232e3dbf7d67d0781ebb5093e23bd08c86e4
Jul 27 22:21:57.299903 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.299703    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:21:57.345364 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:57.345338    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:21:57.346603 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:21:57.346577    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10216a2b_5acf_41d8_bb18_ba11cd182b84.slice/crio-356b40d973e95dcd294aab87369f8f8f7ea11bef7860a9ae0cf15c064aacd207 WatchSource:0}: Error finding container 356b40d973e95dcd294aab87369f8f8f7ea11bef7860a9ae0cf15c064aacd207: Status 404 returned error can't find the container with id 356b40d973e95dcd294aab87369f8f8f7ea11bef7860a9ae0cf15c064aacd207
Jul 27 22:21:58.270429 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:58.270386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" event={"ID":"64dc3969-dcc7-42d7-aaed-ef816b19715f","Type":"ContainerStarted","Data":"16f71df659e8b59c03b9a461a32e232e3dbf7d67d0781ebb5093e23bd08c86e4"}
Jul 27 22:21:58.272546 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:58.272511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" event={"ID":"10216a2b-5acf-41d8-bb18-ba11cd182b84","Type":"ContainerStarted","Data":"356b40d973e95dcd294aab87369f8f8f7ea11bef7860a9ae0cf15c064aacd207"}
Jul 27 22:21:59.278912 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.278798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" event={"ID":"10216a2b-5acf-41d8-bb18-ba11cd182b84","Type":"ContainerStarted","Data":"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"}
Jul 27 22:21:59.279313 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.278995    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:21:59.280295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.280268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" event={"ID":"64dc3969-dcc7-42d7-aaed-ef816b19715f","Type":"ContainerStarted","Data":"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"}
Jul 27 22:21:59.280423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.280324    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:21:59.297933 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.297883    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" podStartSLOduration=1.6364069319999999 podStartE2EDuration="3.297868858s" totalImagesPullingTime="1.661461926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:21:59.295512673 +0000 UTC m=+1629.482353354" watchObservedRunningTime="2026-07-27 22:21:59.297868858 +0000 UTC m=+1629.484709518"
Jul 27 22:21:59.312387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:21:59.312344    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" podStartSLOduration=1.5955945310000001 podStartE2EDuration="3.312331171s" totalImagesPullingTime="1.71673664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:21:59.311504693 +0000 UTC m=+1629.498345351" watchObservedRunningTime="2026-07-27 22:21:59.312331171 +0000 UTC m=+1629.499171829"
Jul 27 22:22:05.848730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:05.848689    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:22:10.290281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:10.290253    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:22:10.290651 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:10.290411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:22:21.496910 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.496874    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"]
Jul 27 22:22:21.500405 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.500388    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:21.503013 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.502991    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:22:21.503119 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.502992    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:22:21.507315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.507278    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"]
Jul 27 22:22:21.600139 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.600114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt\" (UID: \"422e2b79-d60c-4c7f-b68b-b1246b2f19b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:21.700866 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.700831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt\" (UID: \"422e2b79-d60c-4c7f-b68b-b1246b2f19b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:21.701423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.701403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt\" (UID: \"422e2b79-d60c-4c7f-b68b-b1246b2f19b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:21.811834 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.811723    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:21.952274 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:21.952250    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"]
Jul 27 22:22:21.954563 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:22:21.954534    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422e2b79_d60c_4c7f_b68b_b1246b2f19b1.slice/crio-d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199 WatchSource:0}: Error finding container d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199: Status 404 returned error can't find the container with id d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199
Jul 27 22:22:22.395040 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:22.395004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt" event={"ID":"422e2b79-d60c-4c7f-b68b-b1246b2f19b1","Type":"ContainerStarted","Data":"d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199"}
Jul 27 22:22:25.417907 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:25.417865    2578 generic.go:356] "Generic (PLEG): container finished" podID="422e2b79-d60c-4c7f-b68b-b1246b2f19b1" containerID="5052e66e97cb6a10a0bf2c4f14a6c625c69d88a0bc3565d1f5433c72ef698f33" exitCode=0
Jul 27 22:22:25.418310 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:25.417916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt" event={"ID":"422e2b79-d60c-4c7f-b68b-b1246b2f19b1","Type":"ContainerDied","Data":"5052e66e97cb6a10a0bf2c4f14a6c625c69d88a0bc3565d1f5433c72ef698f33"}
Jul 27 22:22:26.560031 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:26.560006    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:26.648390 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:26.648364    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca\") pod \"422e2b79-d60c-4c7f-b68b-b1246b2f19b1\" (UID: \"422e2b79-d60c-4c7f-b68b-b1246b2f19b1\") "
Jul 27 22:22:26.648721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:26.648696    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "422e2b79-d60c-4c7f-b68b-b1246b2f19b1" (UID: "422e2b79-d60c-4c7f-b68b-b1246b2f19b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:22:26.749607 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:26.749579    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422e2b79-d60c-4c7f-b68b-b1246b2f19b1-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:22:27.429099 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:27.429061    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"
Jul 27 22:22:27.429275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:27.429104    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt" event={"ID":"422e2b79-d60c-4c7f-b68b-b1246b2f19b1","Type":"ContainerDied","Data":"d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199"}
Jul 27 22:22:27.429275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:27.429155    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1ef2e56ae4508c07929f036663a30a7719a8c778b1dd3df342930fe20fc2199"
Jul 27 22:22:32.141571 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.141540    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:22:32.142078 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.141852    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" podUID="10216a2b-5acf-41d8-bb18-ba11cd182b84" containerName="payload-processing" containerID="cri-o://223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c" gracePeriod=30
Jul 27 22:22:32.151751 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.151723    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:22:32.152005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.151957    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" podUID="64dc3969-dcc7-42d7-aaed-ef816b19715f" containerName="payload-pre-processing" containerID="cri-o://1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a" gracePeriod=30
Jul 27 22:22:32.436044 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.436014    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:22:32.437940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.437916    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:22:32.453097 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.453067    2578 generic.go:356] "Generic (PLEG): container finished" podID="10216a2b-5acf-41d8-bb18-ba11cd182b84" containerID="223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c" exitCode=0
Jul 27 22:22:32.453228 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.453098    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"
Jul 27 22:22:32.453228 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.453147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" event={"ID":"10216a2b-5acf-41d8-bb18-ba11cd182b84","Type":"ContainerDied","Data":"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"}
Jul 27 22:22:32.453228 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.453201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w" event={"ID":"10216a2b-5acf-41d8-bb18-ba11cd182b84","Type":"ContainerDied","Data":"356b40d973e95dcd294aab87369f8f8f7ea11bef7860a9ae0cf15c064aacd207"}
Jul 27 22:22:32.453404 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.453232    2578 scope.go:117] "RemoveContainer" containerID="223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"
Jul 27 22:22:32.455270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.454852    2578 generic.go:356] "Generic (PLEG): container finished" podID="64dc3969-dcc7-42d7-aaed-ef816b19715f" containerID="1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a" exitCode=0
Jul 27 22:22:32.455270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.454904    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"
Jul 27 22:22:32.455270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.454982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" event={"ID":"64dc3969-dcc7-42d7-aaed-ef816b19715f","Type":"ContainerDied","Data":"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"}
Jul 27 22:22:32.455270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.455031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp" event={"ID":"64dc3969-dcc7-42d7-aaed-ef816b19715f","Type":"ContainerDied","Data":"16f71df659e8b59c03b9a461a32e232e3dbf7d67d0781ebb5093e23bd08c86e4"}
Jul 27 22:22:32.463946 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.463925    2578 scope.go:117] "RemoveContainer" containerID="223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"
Jul 27 22:22:32.464289 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:22:32.464259    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c\": container with ID starting with 223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c not found: ID does not exist" containerID="223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"
Jul 27 22:22:32.464379 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.464292    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c"} err="failed to get container status \"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c\": rpc error: code = NotFound desc = could not find container \"223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c\": container with ID starting with 223bda011afa73407686aca6a6040380a27c584648f5a8d239d69d223158884c not found: ID does not exist"
Jul 27 22:22:32.464379 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.464316    2578 scope.go:117] "RemoveContainer" containerID="1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"
Jul 27 22:22:32.475013 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.474867    2578 scope.go:117] "RemoveContainer" containerID="1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"
Jul 27 22:22:32.475778 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:22:32.475736    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a\": container with ID starting with 1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a not found: ID does not exist" containerID="1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"
Jul 27 22:22:32.475778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.475768    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a"} err="failed to get container status \"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a\": rpc error: code = NotFound desc = could not find container \"1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a\": container with ID starting with 1aae862092172083f753c0e9cac0b29d915a6330417bbb4173a38621f89b424a not found: ID does not exist"
Jul 27 22:22:32.600735 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.600707    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w86jz\" (UniqueName: \"kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz\") pod \"64dc3969-dcc7-42d7-aaed-ef816b19715f\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") "
Jul 27 22:22:32.600735 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.600739    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbk44\" (UniqueName: \"kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44\") pod \"10216a2b-5acf-41d8-bb18-ba11cd182b84\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") "
Jul 27 22:22:32.600992 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.600767    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume\") pod \"64dc3969-dcc7-42d7-aaed-ef816b19715f\" (UID: \"64dc3969-dcc7-42d7-aaed-ef816b19715f\") "
Jul 27 22:22:32.600992 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.600782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume\") pod \"10216a2b-5acf-41d8-bb18-ba11cd182b84\" (UID: \"10216a2b-5acf-41d8-bb18-ba11cd182b84\") "
Jul 27 22:22:32.601186 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.601164    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64dc3969-dcc7-42d7-aaed-ef816b19715f" (UID: "64dc3969-dcc7-42d7-aaed-ef816b19715f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:22:32.601248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.601168    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10216a2b-5acf-41d8-bb18-ba11cd182b84" (UID: "10216a2b-5acf-41d8-bb18-ba11cd182b84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:22:32.602982 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.602959    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz" (OuterVolumeSpecName: "kube-api-access-w86jz") pod "64dc3969-dcc7-42d7-aaed-ef816b19715f" (UID: "64dc3969-dcc7-42d7-aaed-ef816b19715f"). InnerVolumeSpecName "kube-api-access-w86jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:22:32.603067 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.603035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44" (OuterVolumeSpecName: "kube-api-access-nbk44") pod "10216a2b-5acf-41d8-bb18-ba11cd182b84" (UID: "10216a2b-5acf-41d8-bb18-ba11cd182b84"). InnerVolumeSpecName "kube-api-access-nbk44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:22:32.702424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.702340    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w86jz\" (UniqueName: \"kubernetes.io/projected/64dc3969-dcc7-42d7-aaed-ef816b19715f-kube-api-access-w86jz\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:22:32.702424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.702368    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbk44\" (UniqueName: \"kubernetes.io/projected/10216a2b-5acf-41d8-bb18-ba11cd182b84-kube-api-access-nbk44\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:22:32.702424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.702378    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64dc3969-dcc7-42d7-aaed-ef816b19715f-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:22:32.702424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.702387    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10216a2b-5acf-41d8-bb18-ba11cd182b84-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:22:32.776341 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.776312    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:22:32.780464 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.780443    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b8ed72f2-86f647cc44-rn75w"]
Jul 27 22:22:32.791191 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.791168    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:22:32.797542 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:32.797517    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b8ed72f2-8f9975dc6-dj8cp"]
Jul 27 22:22:34.322682 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:34.322641    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10216a2b-5acf-41d8-bb18-ba11cd182b84" path="/var/lib/kubelet/pods/10216a2b-5acf-41d8-bb18-ba11cd182b84/volumes"
Jul 27 22:22:34.323056 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:34.323041    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64dc3969-dcc7-42d7-aaed-ef816b19715f" path="/var/lib/kubelet/pods/64dc3969-dcc7-42d7-aaed-ef816b19715f/volumes"
Jul 27 22:22:35.148607 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:35.148568    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:22:36.577792 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:36.577745    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"]
Jul 27 22:22:36.581708 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:36.581685    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b8ed72f2-fxddt"]
Jul 27 22:22:38.322453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:38.322418    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="422e2b79-d60c-4c7f-b68b-b1246b2f19b1" path="/var/lib/kubelet/pods/422e2b79-d60c-4c7f-b68b-b1246b2f19b1/volumes"
Jul 27 22:22:40.985129 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985087    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:22:40.985691 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985656    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64dc3969-dcc7-42d7-aaed-ef816b19715f" containerName="payload-pre-processing"
Jul 27 22:22:40.985753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985696    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64dc3969-dcc7-42d7-aaed-ef816b19715f" containerName="payload-pre-processing"
Jul 27 22:22:40.985753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985709    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422e2b79-d60c-4c7f-b68b-b1246b2f19b1" containerName="revoke-keys"
Jul 27 22:22:40.985753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985718    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="422e2b79-d60c-4c7f-b68b-b1246b2f19b1" containerName="revoke-keys"
Jul 27 22:22:40.985753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985734    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10216a2b-5acf-41d8-bb18-ba11cd182b84" containerName="payload-processing"
Jul 27 22:22:40.985753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985743    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10216a2b-5acf-41d8-bb18-ba11cd182b84" containerName="payload-processing"
Jul 27 22:22:40.985896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985835    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64dc3969-dcc7-42d7-aaed-ef816b19715f" containerName="payload-pre-processing"
Jul 27 22:22:40.985896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985850    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="422e2b79-d60c-4c7f-b68b-b1246b2f19b1" containerName="revoke-keys"
Jul 27 22:22:40.985896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.985865    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10216a2b-5acf-41d8-bb18-ba11cd182b84" containerName="payload-processing"
Jul 27 22:22:40.991318 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.991296    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:40.996048 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.996029    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-732c17ff-dockercfg-4h58m\""
Jul 27 22:22:40.996414 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:40.996395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-732c17ff\""
Jul 27 22:22:41.008877 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.008852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:22:41.039295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.039269    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:22:41.045693 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.045485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.054976 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.054954    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:22:41.079634 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.079601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5scdn\" (UniqueName: \"kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.079778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.079710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.181082 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.181048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.181242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.181110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfbcv\" (UniqueName: \"kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.181242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.181145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5scdn\" (UniqueName: \"kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.181242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.181162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.181842 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.181823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.189995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.189973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5scdn\" (UniqueName: \"kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn\") pod \"payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.282301 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.282241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.282462 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.282312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfbcv\" (UniqueName: \"kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.282900 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.282879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.291312 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.291287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfbcv\" (UniqueName: \"kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv\") pod \"payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") " pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.301082 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.301059    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:41.358524 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.358493    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:41.454327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.454298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:22:41.457088 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:22:41.457059    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0e41e6_22e6_4cd5_9b17_70b0a06f259b.slice/crio-c24772ea2e697febf1fde5f904c1c8cfe5b5eed6184db2fbbde9524d3aebd022 WatchSource:0}: Error finding container c24772ea2e697febf1fde5f904c1c8cfe5b5eed6184db2fbbde9524d3aebd022: Status 404 returned error can't find the container with id c24772ea2e697febf1fde5f904c1c8cfe5b5eed6184db2fbbde9524d3aebd022
Jul 27 22:22:41.506091 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.506061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" event={"ID":"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b","Type":"ContainerStarted","Data":"c24772ea2e697febf1fde5f904c1c8cfe5b5eed6184db2fbbde9524d3aebd022"}
Jul 27 22:22:41.509750 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:41.509727    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:22:41.511531 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:22:41.511508    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1812f6c_600f_4f8d_a8f8_077c2927acaa.slice/crio-ef5802d0afbe340cc08dbc65ac5a1728d9db1252f1809c61cabafecfaa98abc9 WatchSource:0}: Error finding container ef5802d0afbe340cc08dbc65ac5a1728d9db1252f1809c61cabafecfaa98abc9: Status 404 returned error can't find the container with id ef5802d0afbe340cc08dbc65ac5a1728d9db1252f1809c61cabafecfaa98abc9
Jul 27 22:22:42.512834 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.512798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" event={"ID":"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b","Type":"ContainerStarted","Data":"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"}
Jul 27 22:22:42.513275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.512871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:22:42.514459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.514435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" event={"ID":"a1812f6c-600f-4f8d-a8f8-077c2927acaa","Type":"ContainerStarted","Data":"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"}
Jul 27 22:22:42.514563 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.514486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:42.514563 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.514509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" event={"ID":"a1812f6c-600f-4f8d-a8f8-077c2927acaa","Type":"ContainerStarted","Data":"ef5802d0afbe340cc08dbc65ac5a1728d9db1252f1809c61cabafecfaa98abc9"}
Jul 27 22:22:42.530135 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.530091    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" podStartSLOduration=2.5300764559999998 podStartE2EDuration="2.530076456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:22:42.528789946 +0000 UTC m=+1672.715630607" watchObservedRunningTime="2026-07-27 22:22:42.530076456 +0000 UTC m=+1672.716917114"
Jul 27 22:22:42.545113 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:42.545063    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" podStartSLOduration=1.545049531 podStartE2EDuration="1.545049531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:22:42.544004535 +0000 UTC m=+1672.730845194" watchObservedRunningTime="2026-07-27 22:22:42.545049531 +0000 UTC m=+1672.731890190"
Jul 27 22:22:46.739700 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:46.739649    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:22:53.522392 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:53.522361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:22:53.522790 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:22:53.522439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:23:15.772255 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:15.772165    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:23:15.772852 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:15.772417    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" podUID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" containerName="payload-processing" containerID="cri-o://d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df" gracePeriod=30
Jul 27 22:23:15.784010 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:15.783568    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:23:15.784010 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:15.783804    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" podUID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" containerName="payload-pre-processing" containerID="cri-o://593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0" gracePeriod=30
Jul 27 22:23:16.057742 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.057720    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:23:16.060549 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.060532    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:23:16.170063 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170028    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume\") pod \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") "
Jul 27 22:23:16.170245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170133    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfbcv\" (UniqueName: \"kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv\") pod \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") "
Jul 27 22:23:16.170245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170170    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5scdn\" (UniqueName: \"kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn\") pod \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\" (UID: \"a1812f6c-600f-4f8d-a8f8-077c2927acaa\") "
Jul 27 22:23:16.170245 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170193    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume\") pod \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\" (UID: \"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b\") "
Jul 27 22:23:16.170400 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1812f6c-600f-4f8d-a8f8-077c2927acaa" (UID: "a1812f6c-600f-4f8d-a8f8-077c2927acaa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:23:16.170490 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170472    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1812f6c-600f-4f8d-a8f8-077c2927acaa-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:23:16.170582 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.170562    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" (UID: "4b0e41e6-22e6-4cd5-9b17-70b0a06f259b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:23:16.172354 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.172334    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv" (OuterVolumeSpecName: "kube-api-access-lfbcv") pod "4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" (UID: "4b0e41e6-22e6-4cd5-9b17-70b0a06f259b"). InnerVolumeSpecName "kube-api-access-lfbcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:23:16.172354 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.172345    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn" (OuterVolumeSpecName: "kube-api-access-5scdn") pod "a1812f6c-600f-4f8d-a8f8-077c2927acaa" (UID: "a1812f6c-600f-4f8d-a8f8-077c2927acaa"). InnerVolumeSpecName "kube-api-access-5scdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:23:16.270867 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.270844    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfbcv\" (UniqueName: \"kubernetes.io/projected/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-kube-api-access-lfbcv\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:23:16.270867 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.270865    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5scdn\" (UniqueName: \"kubernetes.io/projected/a1812f6c-600f-4f8d-a8f8-077c2927acaa-kube-api-access-5scdn\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:23:16.270867 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.270874    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:23:16.690427 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.690344    2578 generic.go:356] "Generic (PLEG): container finished" podID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" containerID="d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df" exitCode=0
Jul 27 22:23:16.690427 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.690404    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"
Jul 27 22:23:16.690645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.690395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" event={"ID":"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b","Type":"ContainerDied","Data":"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"}
Jul 27 22:23:16.690645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.690537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h" event={"ID":"4b0e41e6-22e6-4cd5-9b17-70b0a06f259b","Type":"ContainerDied","Data":"c24772ea2e697febf1fde5f904c1c8cfe5b5eed6184db2fbbde9524d3aebd022"}
Jul 27 22:23:16.690645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.690571    2578 scope.go:117] "RemoveContainer" containerID="d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"
Jul 27 22:23:16.692032 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.692004    2578 generic.go:356] "Generic (PLEG): container finished" podID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" containerID="593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0" exitCode=0
Jul 27 22:23:16.692149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.692078    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"
Jul 27 22:23:16.692149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.692091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" event={"ID":"a1812f6c-600f-4f8d-a8f8-077c2927acaa","Type":"ContainerDied","Data":"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"}
Jul 27 22:23:16.692149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.692141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg" event={"ID":"a1812f6c-600f-4f8d-a8f8-077c2927acaa","Type":"ContainerDied","Data":"ef5802d0afbe340cc08dbc65ac5a1728d9db1252f1809c61cabafecfaa98abc9"}
Jul 27 22:23:16.699804 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.699779    2578 scope.go:117] "RemoveContainer" containerID="d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"
Jul 27 22:23:16.700105 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:23:16.700085    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df\": container with ID starting with d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df not found: ID does not exist" containerID="d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"
Jul 27 22:23:16.700171 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.700112    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df"} err="failed to get container status \"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df\": rpc error: code = NotFound desc = could not find container \"d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df\": container with ID starting with d634b147ab2c1f78a29cd4b3d973ba3bc4c806db24b1e67db1d58460fb4e32df not found: ID does not exist"
Jul 27 22:23:16.700171 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.700129    2578 scope.go:117] "RemoveContainer" containerID="593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"
Jul 27 22:23:16.708820 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.708726    2578 scope.go:117] "RemoveContainer" containerID="593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"
Jul 27 22:23:16.709156 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:23:16.709132    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0\": container with ID starting with 593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0 not found: ID does not exist" containerID="593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"
Jul 27 22:23:16.709250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.709180    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0"} err="failed to get container status \"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0\": rpc error: code = NotFound desc = could not find container \"593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0\": container with ID starting with 593005de989d63537e8518516d85e79e680db7c456021d2224983eb30eaa0ae0 not found: ID does not exist"
Jul 27 22:23:16.710122 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.710104    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:23:16.711814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.711796    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-732c17ff-644c4b6ff7-pms9h"]
Jul 27 22:23:16.721403 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.721381    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:23:16.724372 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:16.724354    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-732c17ff-5d546979b-294tg"]
Jul 27 22:23:18.322964 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:18.322932    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" path="/var/lib/kubelet/pods/4b0e41e6-22e6-4cd5-9b17-70b0a06f259b/volumes"
Jul 27 22:23:18.323337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:18.323258    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" path="/var/lib/kubelet/pods/a1812f6c-600f-4f8d-a8f8-077c2927acaa/volumes"
Jul 27 22:23:18.741225 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:18.741198    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:23:24.200956 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.200925    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201289    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" containerName="payload-processing"
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201300    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" containerName="payload-processing"
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201309    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" containerName="payload-pre-processing"
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201315    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" containerName="payload-pre-processing"
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201377    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1812f6c-600f-4f8d-a8f8-077c2927acaa" containerName="payload-pre-processing"
Jul 27 22:23:24.201441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.201386    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0e41e6-22e6-4cd5-9b17-70b0a06f259b" containerName="payload-processing"
Jul 27 22:23:24.204421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.204403    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.208306 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.208280    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-294097f5\""
Jul 27 22:23:24.224646 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.224628    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-294097f5-dockercfg-vbjpw\""
Jul 27 22:23:24.238537 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.238516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:23:24.329877 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.329848    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:23:24.333225 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.333210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.337388 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.337360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97pwc\" (UniqueName: \"kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.337493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.337470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.352796 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.352774    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:23:24.438833 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.438794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdlgg\" (UniqueName: \"kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.438833 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.438836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97pwc\" (UniqueName: \"kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.439080 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.438889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.439134 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.439098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.439777 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.439761    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.458397 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.458332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97pwc\" (UniqueName: \"kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc\") pod \"payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.514871 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.514845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:24.539995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.539966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdlgg\" (UniqueName: \"kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.540117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.540026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.540719 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.540701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.548740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.548716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdlgg\" (UniqueName: \"kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg\") pod \"payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") " pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.643489 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.643455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:24.659635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.659612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:23:24.660925 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:23:24.660886    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678ba3f0_ed23_4efd_a9fb_258a75b2857f.slice/crio-92128bd2b1e71a94b3113779a487f7c415c6f764efaa40cab1083eb1933fb04d WatchSource:0}: Error finding container 92128bd2b1e71a94b3113779a487f7c415c6f764efaa40cab1083eb1933fb04d: Status 404 returned error can't find the container with id 92128bd2b1e71a94b3113779a487f7c415c6f764efaa40cab1083eb1933fb04d
Jul 27 22:23:24.737293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.737221    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" event={"ID":"678ba3f0-ed23-4efd-a9fb-258a75b2857f","Type":"ContainerStarted","Data":"92128bd2b1e71a94b3113779a487f7c415c6f764efaa40cab1083eb1933fb04d"}
Jul 27 22:23:24.994109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:24.994081    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:23:24.995978 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:23:24.995953    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb0f2da_ed5d_45a0_883b_8337baa60e1c.slice/crio-85b9cf5f1e3afdf1feeb2d6854806904348528213b923fb1f3dee0596e9b7726 WatchSource:0}: Error finding container 85b9cf5f1e3afdf1feeb2d6854806904348528213b923fb1f3dee0596e9b7726: Status 404 returned error can't find the container with id 85b9cf5f1e3afdf1feeb2d6854806904348528213b923fb1f3dee0596e9b7726
Jul 27 22:23:25.744005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.743967    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" event={"ID":"ceb0f2da-ed5d-45a0-883b-8337baa60e1c","Type":"ContainerStarted","Data":"ebdf113b8f9f42921660901744b5850d56d11c8428ca359a87a315c6f7ca2b5e"}
Jul 27 22:23:25.744456 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.744028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" event={"ID":"ceb0f2da-ed5d-45a0-883b-8337baa60e1c","Type":"ContainerStarted","Data":"85b9cf5f1e3afdf1feeb2d6854806904348528213b923fb1f3dee0596e9b7726"}
Jul 27 22:23:25.744456 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.744143    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:25.745474 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.745450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" event={"ID":"678ba3f0-ed23-4efd-a9fb-258a75b2857f","Type":"ContainerStarted","Data":"e9f15125f36dcaff350ed09e8cad2105fa9da0e29425677883a13aee58c539cc"}
Jul 27 22:23:25.745534 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.745499    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:25.764811 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.764770    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" podStartSLOduration=1.76475694 podStartE2EDuration="1.76475694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:23:25.763168449 +0000 UTC m=+1715.950009109" watchObservedRunningTime="2026-07-27 22:23:25.76475694 +0000 UTC m=+1715.951597653"
Jul 27 22:23:25.785912 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:25.785863    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" podStartSLOduration=1.785844441 podStartE2EDuration="1.785844441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:23:25.784705303 +0000 UTC m=+1715.971545959" watchObservedRunningTime="2026-07-27 22:23:25.785844441 +0000 UTC m=+1715.972685102"
Jul 27 22:23:30.140466 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:30.140434    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:23:36.752523 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:36.752494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:23:36.752918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:36.752562    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:59.663776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.663728    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:23:59.664293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.664046    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" podUID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" containerName="payload-processing" containerID="cri-o://ebdf113b8f9f42921660901744b5850d56d11c8428ca359a87a315c6f7ca2b5e" gracePeriod=30
Jul 27 22:23:59.670042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.670014    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:23:59.670315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.670251    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" podUID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" containerName="payload-pre-processing" containerID="cri-o://e9f15125f36dcaff350ed09e8cad2105fa9da0e29425677883a13aee58c539cc" gracePeriod=30
Jul 27 22:23:59.920069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.919984    2578 generic.go:356] "Generic (PLEG): container finished" podID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" containerID="ebdf113b8f9f42921660901744b5850d56d11c8428ca359a87a315c6f7ca2b5e" exitCode=0
Jul 27 22:23:59.920230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.920059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" event={"ID":"ceb0f2da-ed5d-45a0-883b-8337baa60e1c","Type":"ContainerDied","Data":"ebdf113b8f9f42921660901744b5850d56d11c8428ca359a87a315c6f7ca2b5e"}
Jul 27 22:23:59.921779 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.921753    2578 generic.go:356] "Generic (PLEG): container finished" podID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" containerID="e9f15125f36dcaff350ed09e8cad2105fa9da0e29425677883a13aee58c539cc" exitCode=0
Jul 27 22:23:59.921883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.921800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" event={"ID":"678ba3f0-ed23-4efd-a9fb-258a75b2857f","Type":"ContainerDied","Data":"e9f15125f36dcaff350ed09e8cad2105fa9da0e29425677883a13aee58c539cc"}
Jul 27 22:23:59.946970 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.946948    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:23:59.949537 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:23:59.949518    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:24:00.050371 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume\") pod \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") "
Jul 27 22:24:00.050574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050398    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97pwc\" (UniqueName: \"kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc\") pod \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") "
Jul 27 22:24:00.050574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050487    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdlgg\" (UniqueName: \"kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg\") pod \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\" (UID: \"ceb0f2da-ed5d-45a0-883b-8337baa60e1c\") "
Jul 27 22:24:00.050574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050524    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume\") pod \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\" (UID: \"678ba3f0-ed23-4efd-a9fb-258a75b2857f\") "
Jul 27 22:24:00.050860 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050769    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ceb0f2da-ed5d-45a0-883b-8337baa60e1c" (UID: "ceb0f2da-ed5d-45a0-883b-8337baa60e1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:24:00.050993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.050971    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "678ba3f0-ed23-4efd-a9fb-258a75b2857f" (UID: "678ba3f0-ed23-4efd-a9fb-258a75b2857f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:24:00.052721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.052697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg" (OuterVolumeSpecName: "kube-api-access-hdlgg") pod "ceb0f2da-ed5d-45a0-883b-8337baa60e1c" (UID: "ceb0f2da-ed5d-45a0-883b-8337baa60e1c"). InnerVolumeSpecName "kube-api-access-hdlgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:24:00.052825 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.052759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc" (OuterVolumeSpecName: "kube-api-access-97pwc") pod "678ba3f0-ed23-4efd-a9fb-258a75b2857f" (UID: "678ba3f0-ed23-4efd-a9fb-258a75b2857f"). InnerVolumeSpecName "kube-api-access-97pwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:24:00.151851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.151818    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdlgg\" (UniqueName: \"kubernetes.io/projected/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-kube-api-access-hdlgg\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:00.151851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.151846    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/678ba3f0-ed23-4efd-a9fb-258a75b2857f-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:00.151851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.151855    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ceb0f2da-ed5d-45a0-883b-8337baa60e1c-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:00.152071 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.151866    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97pwc\" (UniqueName: \"kubernetes.io/projected/678ba3f0-ed23-4efd-a9fb-258a75b2857f-kube-api-access-97pwc\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:00.927732 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.927703    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"
Jul 27 22:24:00.928183 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.927702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7" event={"ID":"ceb0f2da-ed5d-45a0-883b-8337baa60e1c","Type":"ContainerDied","Data":"85b9cf5f1e3afdf1feeb2d6854806904348528213b923fb1f3dee0596e9b7726"}
Jul 27 22:24:00.928183 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.927845    2578 scope.go:117] "RemoveContainer" containerID="ebdf113b8f9f42921660901744b5850d56d11c8428ca359a87a315c6f7ca2b5e"
Jul 27 22:24:00.929225 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.929156    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"
Jul 27 22:24:00.929295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.929157    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94" event={"ID":"678ba3f0-ed23-4efd-a9fb-258a75b2857f","Type":"ContainerDied","Data":"92128bd2b1e71a94b3113779a487f7c415c6f764efaa40cab1083eb1933fb04d"}
Jul 27 22:24:00.936786 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.936740    2578 scope.go:117] "RemoveContainer" containerID="e9f15125f36dcaff350ed09e8cad2105fa9da0e29425677883a13aee58c539cc"
Jul 27 22:24:00.945684 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.945648    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:24:00.950573 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.950547    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-294097f5-7756cfb55b-lk7j7"]
Jul 27 22:24:00.970244 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.970220    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:24:00.976946 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:00.976928    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-294097f5-6fb4c8cf4d-2hq94"]
Jul 27 22:24:02.322557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:02.322519    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" path="/var/lib/kubelet/pods/678ba3f0-ed23-4efd-a9fb-258a75b2857f/volumes"
Jul 27 22:24:02.322949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:02.322896    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" path="/var/lib/kubelet/pods/ceb0f2da-ed5d-45a0-883b-8337baa60e1c/volumes"
Jul 27 22:24:02.647445 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:02.647363    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:24:09.583190 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583153    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:09.583722 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583703    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" containerName="payload-processing"
Jul 27 22:24:09.583817 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583727    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" containerName="payload-processing"
Jul 27 22:24:09.583817 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583749    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" containerName="payload-pre-processing"
Jul 27 22:24:09.583817 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583758    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" containerName="payload-pre-processing"
Jul 27 22:24:09.583973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583877    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="678ba3f0-ed23-4efd-a9fb-258a75b2857f" containerName="payload-pre-processing"
Jul 27 22:24:09.583973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.583894    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb0f2da-ed5d-45a0-883b-8337baa60e1c" containerName="payload-processing"
Jul 27 22:24:09.589124 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.589102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.593430 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.593403    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-858f23ab\""
Jul 27 22:24:09.602469 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.602446    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:09.629918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.629891    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:09.633308 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.633289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.633953 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.633934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.634065 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.634022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt247\" (UniqueName: \"kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.636221 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.636204    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-858f23ab-dockercfg-vvwhq\""
Jul 27 22:24:09.649026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.649003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:09.735359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.735325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt247\" (UniqueName: \"kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.735535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.735365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x8hf\" (UniqueName: \"kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.735535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.735409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.735535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.735447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.736179 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.736160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.744372 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.744340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt247\" (UniqueName: \"kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247\") pod \"payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.836846 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.836780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x8hf\" (UniqueName: \"kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.836846 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.836819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.837428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.837412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.845347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.845327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x8hf\" (UniqueName: \"kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf\") pod \"payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") " pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:09.902904 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.902875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:09.945160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:09.944876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:10.056100 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.056000    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:10.059098 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:24:10.059052    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7196e17c_70ca_4fe1_8bfe_3654c3390563.slice/crio-339522a323d187817c21a97a19b82044a259f00af8000e825c0b40e478e2e994 WatchSource:0}: Error finding container 339522a323d187817c21a97a19b82044a259f00af8000e825c0b40e478e2e994: Status 404 returned error can't find the container with id 339522a323d187817c21a97a19b82044a259f00af8000e825c0b40e478e2e994
Jul 27 22:24:10.109272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.109242    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:10.112957 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:24:10.112918    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5080cecc_4556_4c0b_80a8_fa9e368d24ae.slice/crio-c60842136a08412b0bafe6f91bc3e0b4cd3dd2aa6820f50321f0958bffd5c3de WatchSource:0}: Error finding container c60842136a08412b0bafe6f91bc3e0b4cd3dd2aa6820f50321f0958bffd5c3de: Status 404 returned error can't find the container with id c60842136a08412b0bafe6f91bc3e0b4cd3dd2aa6820f50321f0958bffd5c3de
Jul 27 22:24:10.984154 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.984114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" event={"ID":"7196e17c-70ca-4fe1-8bfe-3654c3390563","Type":"ContainerStarted","Data":"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"}
Jul 27 22:24:10.984612 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.984176    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" event={"ID":"7196e17c-70ca-4fe1-8bfe-3654c3390563","Type":"ContainerStarted","Data":"339522a323d187817c21a97a19b82044a259f00af8000e825c0b40e478e2e994"}
Jul 27 22:24:10.984612 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.984417    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:10.986430 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.986400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" event={"ID":"5080cecc-4556-4c0b-80a8-fa9e368d24ae","Type":"ContainerStarted","Data":"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"}
Jul 27 22:24:10.986582 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.986453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" event={"ID":"5080cecc-4556-4c0b-80a8-fa9e368d24ae","Type":"ContainerStarted","Data":"c60842136a08412b0bafe6f91bc3e0b4cd3dd2aa6820f50321f0958bffd5c3de"}
Jul 27 22:24:10.986582 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:10.986483    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:11.004746 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:11.004700    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" podStartSLOduration=2.004665082 podStartE2EDuration="2.004665082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:24:11.003085023 +0000 UTC m=+1761.189925682" watchObservedRunningTime="2026-07-27 22:24:11.004665082 +0000 UTC m=+1761.191525022"
Jul 27 22:24:11.023473 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:11.023421    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" podStartSLOduration=2.023405149 podStartE2EDuration="2.023405149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:24:11.021077965 +0000 UTC m=+1761.207918631" watchObservedRunningTime="2026-07-27 22:24:11.023405149 +0000 UTC m=+1761.210245807"
Jul 27 22:24:13.948873 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:13.948838    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:24:21.994123 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:21.994092    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:21.994532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:21.994280    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:44.195280 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.195200    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:44.195827 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.195423    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" podUID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" containerName="payload-processing" containerID="cri-o://9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525" gracePeriod=30
Jul 27 22:24:44.216899 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.216872    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:44.217109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.217087    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" podUID="7196e17c-70ca-4fe1-8bfe-3654c3390563" containerName="payload-pre-processing" containerID="cri-o://48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040" gracePeriod=30
Jul 27 22:24:44.498275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.498253    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:44.501022 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.501003    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:44.542288 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume\") pod \"7196e17c-70ca-4fe1-8bfe-3654c3390563\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") "
Jul 27 22:24:44.542423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542303    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume\") pod \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") "
Jul 27 22:24:44.542423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt247\" (UniqueName: \"kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247\") pod \"7196e17c-70ca-4fe1-8bfe-3654c3390563\" (UID: \"7196e17c-70ca-4fe1-8bfe-3654c3390563\") "
Jul 27 22:24:44.542423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542421    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x8hf\" (UniqueName: \"kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf\") pod \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\" (UID: \"5080cecc-4556-4c0b-80a8-fa9e368d24ae\") "
Jul 27 22:24:44.542714 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542688    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7196e17c-70ca-4fe1-8bfe-3654c3390563" (UID: "7196e17c-70ca-4fe1-8bfe-3654c3390563"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:24:44.542714 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.542701    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5080cecc-4556-4c0b-80a8-fa9e368d24ae" (UID: "5080cecc-4556-4c0b-80a8-fa9e368d24ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:24:44.544720 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.544695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247" (OuterVolumeSpecName: "kube-api-access-jt247") pod "7196e17c-70ca-4fe1-8bfe-3654c3390563" (UID: "7196e17c-70ca-4fe1-8bfe-3654c3390563"). InnerVolumeSpecName "kube-api-access-jt247". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:24:44.544839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.544714    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf" (OuterVolumeSpecName: "kube-api-access-8x8hf") pod "5080cecc-4556-4c0b-80a8-fa9e368d24ae" (UID: "5080cecc-4556-4c0b-80a8-fa9e368d24ae"). InnerVolumeSpecName "kube-api-access-8x8hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:24:44.643869 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.643847    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5080cecc-4556-4c0b-80a8-fa9e368d24ae-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:44.643869 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.643869    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt247\" (UniqueName: \"kubernetes.io/projected/7196e17c-70ca-4fe1-8bfe-3654c3390563-kube-api-access-jt247\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:44.644037 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.643878    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x8hf\" (UniqueName: \"kubernetes.io/projected/5080cecc-4556-4c0b-80a8-fa9e368d24ae-kube-api-access-8x8hf\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:44.644037 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:44.643887    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7196e17c-70ca-4fe1-8bfe-3654c3390563-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:24:45.164798 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.164763    2578 generic.go:356] "Generic (PLEG): container finished" podID="7196e17c-70ca-4fe1-8bfe-3654c3390563" containerID="48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040" exitCode=0
Jul 27 22:24:45.165005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.164810    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"
Jul 27 22:24:45.165005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.164855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" event={"ID":"7196e17c-70ca-4fe1-8bfe-3654c3390563","Type":"ContainerDied","Data":"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"}
Jul 27 22:24:45.165005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.164952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q" event={"ID":"7196e17c-70ca-4fe1-8bfe-3654c3390563","Type":"ContainerDied","Data":"339522a323d187817c21a97a19b82044a259f00af8000e825c0b40e478e2e994"}
Jul 27 22:24:45.165005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.164984    2578 scope.go:117] "RemoveContainer" containerID="48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"
Jul 27 22:24:45.166189 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.166166    2578 generic.go:356] "Generic (PLEG): container finished" podID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" containerID="9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525" exitCode=0
Jul 27 22:24:45.166257 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.166194    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" event={"ID":"5080cecc-4556-4c0b-80a8-fa9e368d24ae","Type":"ContainerDied","Data":"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"}
Jul 27 22:24:45.166257 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.166228    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"
Jul 27 22:24:45.166257 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.166238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9" event={"ID":"5080cecc-4556-4c0b-80a8-fa9e368d24ae","Type":"ContainerDied","Data":"c60842136a08412b0bafe6f91bc3e0b4cd3dd2aa6820f50321f0958bffd5c3de"}
Jul 27 22:24:45.176866 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.176850    2578 scope.go:117] "RemoveContainer" containerID="48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"
Jul 27 22:24:45.177167 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:24:45.177142    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040\": container with ID starting with 48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040 not found: ID does not exist" containerID="48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"
Jul 27 22:24:45.177250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.177201    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040"} err="failed to get container status \"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040\": rpc error: code = NotFound desc = could not find container \"48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040\": container with ID starting with 48717d5b39fc99eff1822b87a815fa4c86b741ff955f16660e8ec7f8c1252040 not found: ID does not exist"
Jul 27 22:24:45.177250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.177227    2578 scope.go:117] "RemoveContainer" containerID="9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"
Jul 27 22:24:45.185294 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.185278    2578 scope.go:117] "RemoveContainer" containerID="9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"
Jul 27 22:24:45.185529 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:24:45.185513    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525\": container with ID starting with 9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525 not found: ID does not exist" containerID="9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"
Jul 27 22:24:45.185605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.185535    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525"} err="failed to get container status \"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525\": rpc error: code = NotFound desc = could not find container \"9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525\": container with ID starting with 9b4d138bf2d6b7700f65f68f3b07cb5ba76fb6653aade02b989dde9f58f83525 not found: ID does not exist"
Jul 27 22:24:45.189006 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.188983    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:45.192901 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.192880    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-858f23ab-79c6fb794d-g785q"]
Jul 27 22:24:45.204272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.204249    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:45.208729 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:45.208711    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-858f23ab-689b4c499d-kq5m9"]
Jul 27 22:24:46.323267 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:46.323235    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" path="/var/lib/kubelet/pods/5080cecc-4556-4c0b-80a8-fa9e368d24ae/volumes"
Jul 27 22:24:46.323640 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:46.323577    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7196e17c-70ca-4fe1-8bfe-3654c3390563" path="/var/lib/kubelet/pods/7196e17c-70ca-4fe1-8bfe-3654c3390563/volumes"
Jul 27 22:24:46.951540 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:46.951509    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:24:50.296635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:50.296606    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:24:50.301255 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:24:50.301231    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:26:15.801900 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:15.801820    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:26:25.947723 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:25.947684    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:26:31.891859 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.891818    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:31.892389 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892372    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" containerName="payload-processing"
Jul 27 22:26:31.892438 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892391    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" containerName="payload-processing"
Jul 27 22:26:31.892438 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892409    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7196e17c-70ca-4fe1-8bfe-3654c3390563" containerName="payload-pre-processing"
Jul 27 22:26:31.892438 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892415    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7196e17c-70ca-4fe1-8bfe-3654c3390563" containerName="payload-pre-processing"
Jul 27 22:26:31.892536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892470    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7196e17c-70ca-4fe1-8bfe-3654c3390563" containerName="payload-pre-processing"
Jul 27 22:26:31.892536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.892480    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5080cecc-4556-4c0b-80a8-fa9e368d24ae" containerName="payload-processing"
Jul 27 22:26:31.895511 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.895497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:31.907882 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.907860    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-92c78b5c-dockercfg-86ztq\""
Jul 27 22:26:31.908007 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.907906    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-92c78b5c\""
Jul 27 22:26:31.932446 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.932417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:31.963686 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.963639    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:31.967059 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.967040    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:31.975915 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.975892    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:31.995176 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.995139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:31.995317 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.995186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:31.995317 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.995292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jjff\" (UniqueName: \"kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:31.995452 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:31.995330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xn9b\" (UniqueName: \"kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.096690 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.096634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xn9b\" (UniqueName: \"kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.096855 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.096704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.096855 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.096748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.096855 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.096829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jjff\" (UniqueName: \"kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.097665 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.097642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.097752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.097642    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.105838 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.105804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xn9b\" (UniqueName: \"kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b\") pod \"payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.106608 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.106587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jjff\" (UniqueName: \"kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff\") pod \"payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") " pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.206254 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.206171    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.277306 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.277265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.352153 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.352123    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:32.354981 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:26:32.354844    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603f86f1_c230_45e4_a59b_bb1a3944edb0.slice/crio-7af933d1346f34d9a16ef6f23634bf811e664342af550bfc761925c31ad211d3 WatchSource:0}: Error finding container 7af933d1346f34d9a16ef6f23634bf811e664342af550bfc761925c31ad211d3: Status 404 returned error can't find the container with id 7af933d1346f34d9a16ef6f23634bf811e664342af550bfc761925c31ad211d3
Jul 27 22:26:32.430626 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.430599    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:32.432385 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:26:32.432358    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236f994b_a2f6_4be0_acec_b393f427aa53.slice/crio-617b5284e5b9f1b02b53de44834e7b0aae1a7c154bf548cd3a896d47f391f53d WatchSource:0}: Error finding container 617b5284e5b9f1b02b53de44834e7b0aae1a7c154bf548cd3a896d47f391f53d: Status 404 returned error can't find the container with id 617b5284e5b9f1b02b53de44834e7b0aae1a7c154bf548cd3a896d47f391f53d
Jul 27 22:26:32.688421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.688180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" event={"ID":"603f86f1-c230-45e4-a59b-bb1a3944edb0","Type":"ContainerStarted","Data":"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"}
Jul 27 22:26:32.688421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.688238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" event={"ID":"603f86f1-c230-45e4-a59b-bb1a3944edb0","Type":"ContainerStarted","Data":"7af933d1346f34d9a16ef6f23634bf811e664342af550bfc761925c31ad211d3"}
Jul 27 22:26:32.688421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.688341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:32.689993 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.689966    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" event={"ID":"236f994b-a2f6-4be0-acec-b393f427aa53","Type":"ContainerStarted","Data":"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"}
Jul 27 22:26:32.690111 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.690007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" event={"ID":"236f994b-a2f6-4be0-acec-b393f427aa53","Type":"ContainerStarted","Data":"617b5284e5b9f1b02b53de44834e7b0aae1a7c154bf548cd3a896d47f391f53d"}
Jul 27 22:26:32.690111 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.690093    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:32.706698 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.706638    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" podStartSLOduration=1.706625302 podStartE2EDuration="1.706625302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:26:32.704916685 +0000 UTC m=+1902.891757375" watchObservedRunningTime="2026-07-27 22:26:32.706625302 +0000 UTC m=+1902.893465983"
Jul 27 22:26:32.722772 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:32.722661    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" podStartSLOduration=1.7226448749999999 podStartE2EDuration="1.722644875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:26:32.721428572 +0000 UTC m=+1902.908269233" watchObservedRunningTime="2026-07-27 22:26:32.722644875 +0000 UTC m=+1902.909485533"
Jul 27 22:26:37.943350 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:37.943318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:26:43.697868 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:43.697839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:43.698272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:43.697911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:53.057353 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.057317    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:53.057783 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.057625    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" podUID="603f86f1-c230-45e4-a59b-bb1a3944edb0" containerName="payload-processing" containerID="cri-o://dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97" gracePeriod=30
Jul 27 22:26:53.064788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.064758    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:53.065289 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.065235    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" podUID="236f994b-a2f6-4be0-acec-b393f427aa53" containerName="payload-pre-processing" containerID="cri-o://ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35" gracePeriod=30
Jul 27 22:26:53.344275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.344253    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:53.347065 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.347047    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:53.493138 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493099    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume\") pod \"603f86f1-c230-45e4-a59b-bb1a3944edb0\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") "
Jul 27 22:26:53.493138 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xn9b\" (UniqueName: \"kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b\") pod \"236f994b-a2f6-4be0-acec-b393f427aa53\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") "
Jul 27 22:26:53.493413 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jjff\" (UniqueName: \"kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff\") pod \"603f86f1-c230-45e4-a59b-bb1a3944edb0\" (UID: \"603f86f1-c230-45e4-a59b-bb1a3944edb0\") "
Jul 27 22:26:53.493413 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493212    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume\") pod \"236f994b-a2f6-4be0-acec-b393f427aa53\" (UID: \"236f994b-a2f6-4be0-acec-b393f427aa53\") "
Jul 27 22:26:53.493599 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "603f86f1-c230-45e4-a59b-bb1a3944edb0" (UID: "603f86f1-c230-45e4-a59b-bb1a3944edb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:26:53.493744 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.493694    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "236f994b-a2f6-4be0-acec-b393f427aa53" (UID: "236f994b-a2f6-4be0-acec-b393f427aa53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:26:53.495588 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.495556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b" (OuterVolumeSpecName: "kube-api-access-6xn9b") pod "236f994b-a2f6-4be0-acec-b393f427aa53" (UID: "236f994b-a2f6-4be0-acec-b393f427aa53"). InnerVolumeSpecName "kube-api-access-6xn9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:53.495588 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.495576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff" (OuterVolumeSpecName: "kube-api-access-7jjff") pod "603f86f1-c230-45e4-a59b-bb1a3944edb0" (UID: "603f86f1-c230-45e4-a59b-bb1a3944edb0"). InnerVolumeSpecName "kube-api-access-7jjff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:26:53.594821 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.594739    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jjff\" (UniqueName: \"kubernetes.io/projected/603f86f1-c230-45e4-a59b-bb1a3944edb0-kube-api-access-7jjff\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:26:53.594821 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.594771    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/236f994b-a2f6-4be0-acec-b393f427aa53-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:26:53.594821 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.594780    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/603f86f1-c230-45e4-a59b-bb1a3944edb0-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:26:53.594821 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.594789    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xn9b\" (UniqueName: \"kubernetes.io/projected/236f994b-a2f6-4be0-acec-b393f427aa53-kube-api-access-6xn9b\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:26:53.793995 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.793953    2578 generic.go:356] "Generic (PLEG): container finished" podID="603f86f1-c230-45e4-a59b-bb1a3944edb0" containerID="dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97" exitCode=0
Jul 27 22:26:53.794185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.794015    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"
Jul 27 22:26:53.794185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.794049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" event={"ID":"603f86f1-c230-45e4-a59b-bb1a3944edb0","Type":"ContainerDied","Data":"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"}
Jul 27 22:26:53.794185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.794121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl" event={"ID":"603f86f1-c230-45e4-a59b-bb1a3944edb0","Type":"ContainerDied","Data":"7af933d1346f34d9a16ef6f23634bf811e664342af550bfc761925c31ad211d3"}
Jul 27 22:26:53.794185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.794152    2578 scope.go:117] "RemoveContainer" containerID="dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"
Jul 27 22:26:53.795461 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.795438    2578 generic.go:356] "Generic (PLEG): container finished" podID="236f994b-a2f6-4be0-acec-b393f427aa53" containerID="ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35" exitCode=0
Jul 27 22:26:53.795549 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.795475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" event={"ID":"236f994b-a2f6-4be0-acec-b393f427aa53","Type":"ContainerDied","Data":"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"}
Jul 27 22:26:53.795549 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.795537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k" event={"ID":"236f994b-a2f6-4be0-acec-b393f427aa53","Type":"ContainerDied","Data":"617b5284e5b9f1b02b53de44834e7b0aae1a7c154bf548cd3a896d47f391f53d"}
Jul 27 22:26:53.795654 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.795486    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"
Jul 27 22:26:53.807308 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.807290    2578 scope.go:117] "RemoveContainer" containerID="dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"
Jul 27 22:26:53.807509 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:26:53.807491    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97\": container with ID starting with dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97 not found: ID does not exist" containerID="dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"
Jul 27 22:26:53.807569 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.807517    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97"} err="failed to get container status \"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97\": rpc error: code = NotFound desc = could not find container \"dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97\": container with ID starting with dd77d92ddd2141727f4a6eb229be5a52f4a257745ebdb06f6ce840dc8ed4ef97 not found: ID does not exist"
Jul 27 22:26:53.807569 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.807551    2578 scope.go:117] "RemoveContainer" containerID="ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"
Jul 27 22:26:53.815499 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.815481    2578 scope.go:117] "RemoveContainer" containerID="ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"
Jul 27 22:26:53.815747 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:26:53.815727    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35\": container with ID starting with ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35 not found: ID does not exist" containerID="ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"
Jul 27 22:26:53.815813 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.815753    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35"} err="failed to get container status \"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35\": rpc error: code = NotFound desc = could not find container \"ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35\": container with ID starting with ed857b856e6da151d757b507d4080779ca5e0f684d4175779e7b6bedfbc75e35 not found: ID does not exist"
Jul 27 22:26:53.821387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.821367    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:53.824869 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.824850    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c78b5c-59f8d5fc46-6qcbl"]
Jul 27 22:26:53.836073 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.836050    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:53.839455 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:53.839434    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c78b5c-8464c559d4-46g5k"]
Jul 27 22:26:54.323115 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:54.323081    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="236f994b-a2f6-4be0-acec-b393f427aa53" path="/var/lib/kubelet/pods/236f994b-a2f6-4be0-acec-b393f427aa53/volumes"
Jul 27 22:26:54.323500 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:54.323417    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="603f86f1-c230-45e4-a59b-bb1a3944edb0" path="/var/lib/kubelet/pods/603f86f1-c230-45e4-a59b-bb1a3944edb0/volumes"
Jul 27 22:26:56.047695 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:26:56.047638    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:27:28.988336 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988302    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:27:28.988949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988743    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="603f86f1-c230-45e4-a59b-bb1a3944edb0" containerName="payload-processing"
Jul 27 22:27:28.988949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988756    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="603f86f1-c230-45e4-a59b-bb1a3944edb0" containerName="payload-processing"
Jul 27 22:27:28.988949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988766    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="236f994b-a2f6-4be0-acec-b393f427aa53" containerName="payload-pre-processing"
Jul 27 22:27:28.988949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988784    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="236f994b-a2f6-4be0-acec-b393f427aa53" containerName="payload-pre-processing"
Jul 27 22:27:28.988949 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.988862    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="236f994b-a2f6-4be0-acec-b393f427aa53" containerName="payload-pre-processing"
Jul 27 22:27:28.989281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.989261    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="603f86f1-c230-45e4-a59b-bb1a3944edb0" containerName="payload-processing"
Jul 27 22:27:28.999753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:28.999720    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.002963 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.002938    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d2650623-dockercfg-lcgdw\""
Jul 27 22:27:29.003216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.003192    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d2650623\""
Jul 27 22:27:29.003809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.003779    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:27:29.006052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.006032    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:27:29.009519 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.009500    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.015158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.015137    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:27:29.098491 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.098457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tkcj\" (UniqueName: \"kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.098730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.098504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.098730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.098546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.098730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.098588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gskps\" (UniqueName: \"kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.199622 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.199585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tkcj\" (UniqueName: \"kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.199814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.199633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.199814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.199708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.199814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.199753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gskps\" (UniqueName: \"kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.200384 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.200362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.200642 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.200620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.209004 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.208975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tkcj\" (UniqueName: \"kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj\") pod \"payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.209099 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.209038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gskps\" (UniqueName: \"kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps\") pod \"payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") " pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.311825 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.311752    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.321844 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.321717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.492272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.492247    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:27:29.495202 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:27:29.495175    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00965727_aa42_4945_b20e_c96f1262a5df.slice/crio-b59c33c25b2bf76e46af8adf95f6ac6881af2466278bf139875b915adac27ebe WatchSource:0}: Error finding container b59c33c25b2bf76e46af8adf95f6ac6881af2466278bf139875b915adac27ebe: Status 404 returned error can't find the container with id b59c33c25b2bf76e46af8adf95f6ac6881af2466278bf139875b915adac27ebe
Jul 27 22:27:29.663840 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.663816    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:27:29.665547 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:27:29.665523    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57701f93_f2e2_498d_93cf_f45099099aa2.slice/crio-8ddebbdadbdb6d377970bc415a2561cef75217b3067d59358beda92fbca54db6 WatchSource:0}: Error finding container 8ddebbdadbdb6d377970bc415a2561cef75217b3067d59358beda92fbca54db6: Status 404 returned error can't find the container with id 8ddebbdadbdb6d377970bc415a2561cef75217b3067d59358beda92fbca54db6
Jul 27 22:27:29.974691 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.974575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" event={"ID":"00965727-aa42-4945-b20e-c96f1262a5df","Type":"ContainerStarted","Data":"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"}
Jul 27 22:27:29.974691 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.974627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" event={"ID":"00965727-aa42-4945-b20e-c96f1262a5df","Type":"ContainerStarted","Data":"b59c33c25b2bf76e46af8adf95f6ac6881af2466278bf139875b915adac27ebe"}
Jul 27 22:27:29.974911 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.974702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:29.976107 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.976084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" event={"ID":"57701f93-f2e2-498d-93cf-f45099099aa2","Type":"ContainerStarted","Data":"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"}
Jul 27 22:27:29.976202 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.976123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:29.976202 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.976140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" event={"ID":"57701f93-f2e2-498d-93cf-f45099099aa2","Type":"ContainerStarted","Data":"8ddebbdadbdb6d377970bc415a2561cef75217b3067d59358beda92fbca54db6"}
Jul 27 22:27:29.990177 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:29.990121    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" podStartSLOduration=1.9901041940000002 podStartE2EDuration="1.990104194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:27:29.989768593 +0000 UTC m=+1960.176609252" watchObservedRunningTime="2026-07-27 22:27:29.990104194 +0000 UTC m=+1960.176944845"
Jul 27 22:27:30.006250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:30.006190    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" podStartSLOduration=2.006176002 podStartE2EDuration="2.006176002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:27:30.004988655 +0000 UTC m=+1960.191829316" watchObservedRunningTime="2026-07-27 22:27:30.006176002 +0000 UTC m=+1960.193016643"
Jul 27 22:27:34.650185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:34.650146    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:27:40.984383 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:40.984350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:27:40.984883 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:40.984440    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:27:41.551161 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.551122    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:27:41.554909 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.554889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.559617 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.559591    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d2650623\"/\"openshift-service-ca.crt\""
Jul 27 22:27:41.559776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.559603    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d2650623\"/\"e2e-lifecycle-model-d2650623-kserve-self-signed-certs\""
Jul 27 22:27:41.559776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.559656    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d2650623\"/\"default-dockercfg-pvt8s\""
Jul 27 22:27:41.559776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.559687    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d2650623\"/\"kube-root-ca.crt\""
Jul 27 22:27:41.572916 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.572888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:27:41.617227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617195    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.617227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.617450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.617450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.617450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617408    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.617450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.617444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5s9f\" (UniqueName: \"kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718527 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718527 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.718809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.718690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5s9f\" (UniqueName: \"kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.719050 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.719029    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.719050 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.719042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.719139 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.719100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.720939 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.720919    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.721194 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.721176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.729404 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.729381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5s9f\" (UniqueName: \"kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f\") pod \"e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") " pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:41.865364 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:41.865280    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:42.006989 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:42.006961    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:27:42.008728 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:27:42.008703    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode05be0af_4e32_48fb_9e37_71f935e7f2fa.slice/crio-05c2ed9d23af3971ddc7864ca1264748ae65aaa55760647d048a8383d87c9cc7 WatchSource:0}: Error finding container 05c2ed9d23af3971ddc7864ca1264748ae65aaa55760647d048a8383d87c9cc7: Status 404 returned error can't find the container with id 05c2ed9d23af3971ddc7864ca1264748ae65aaa55760647d048a8383d87c9cc7
Jul 27 22:27:42.040875 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:42.040846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" event={"ID":"e05be0af-4e32-48fb-9e37-71f935e7f2fa","Type":"ContainerStarted","Data":"05c2ed9d23af3971ddc7864ca1264748ae65aaa55760647d048a8383d87c9cc7"}
Jul 27 22:27:43.047770 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:43.047732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" event={"ID":"e05be0af-4e32-48fb-9e37-71f935e7f2fa","Type":"ContainerStarted","Data":"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"}
Jul 27 22:27:43.065050 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:43.064994    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" podStartSLOduration=2.064977848 podStartE2EDuration="2.064977848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:27:43.063252758 +0000 UTC m=+1973.250093420" watchObservedRunningTime="2026-07-27 22:27:43.064977848 +0000 UTC m=+1973.251818507"
Jul 27 22:27:44.741905 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:44.741868    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:27:51.865503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:51.865472    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:51.865503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:51.865511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:51.877823 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:51.877797    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:27:52.108284 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:27:52.108252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:28:02.460921 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:02.460880    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:28:21.439762 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:21.439725    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:28:22.815464 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:22.815426    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:28:22.815958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:22.815699    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" podUID="00965727-aa42-4945-b20e-c96f1262a5df" containerName="payload-processing" containerID="cri-o://4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35" gracePeriod=30
Jul 27 22:28:22.824194 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:22.824164    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:28:22.824419 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:22.824367    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" podUID="57701f93-f2e2-498d-93cf-f45099099aa2" containerName="payload-pre-processing" containerID="cri-o://bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1" gracePeriod=30
Jul 27 22:28:23.071205 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.071140    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:28:23.114110 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.114082    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:28:23.182162 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182132    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume\") pod \"57701f93-f2e2-498d-93cf-f45099099aa2\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") "
Jul 27 22:28:23.182321 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tkcj\" (UniqueName: \"kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj\") pod \"57701f93-f2e2-498d-93cf-f45099099aa2\" (UID: \"57701f93-f2e2-498d-93cf-f45099099aa2\") "
Jul 27 22:28:23.182321 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182215    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gskps\" (UniqueName: \"kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps\") pod \"00965727-aa42-4945-b20e-c96f1262a5df\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") "
Jul 27 22:28:23.182398 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182327    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume\") pod \"00965727-aa42-4945-b20e-c96f1262a5df\" (UID: \"00965727-aa42-4945-b20e-c96f1262a5df\") "
Jul 27 22:28:23.182507 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57701f93-f2e2-498d-93cf-f45099099aa2" (UID: "57701f93-f2e2-498d-93cf-f45099099aa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:23.182724 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00965727-aa42-4945-b20e-c96f1262a5df" (UID: "00965727-aa42-4945-b20e-c96f1262a5df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:28:23.182904 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.182827    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57701f93-f2e2-498d-93cf-f45099099aa2-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:23.184536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.184507    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps" (OuterVolumeSpecName: "kube-api-access-gskps") pod "00965727-aa42-4945-b20e-c96f1262a5df" (UID: "00965727-aa42-4945-b20e-c96f1262a5df"). InnerVolumeSpecName "kube-api-access-gskps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:23.184819 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.184800    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj" (OuterVolumeSpecName: "kube-api-access-4tkcj") pod "57701f93-f2e2-498d-93cf-f45099099aa2" (UID: "57701f93-f2e2-498d-93cf-f45099099aa2"). InnerVolumeSpecName "kube-api-access-4tkcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:23.264635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.264597    2578 generic.go:356] "Generic (PLEG): container finished" podID="00965727-aa42-4945-b20e-c96f1262a5df" containerID="4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35" exitCode=0
Jul 27 22:28:23.264822 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.264653    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"
Jul 27 22:28:23.264822 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.264695    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" event={"ID":"00965727-aa42-4945-b20e-c96f1262a5df","Type":"ContainerDied","Data":"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"}
Jul 27 22:28:23.264822 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.264789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp" event={"ID":"00965727-aa42-4945-b20e-c96f1262a5df","Type":"ContainerDied","Data":"b59c33c25b2bf76e46af8adf95f6ac6881af2466278bf139875b915adac27ebe"}
Jul 27 22:28:23.264822 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.264815    2578 scope.go:117] "RemoveContainer" containerID="4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"
Jul 27 22:28:23.265985 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.265966    2578 generic.go:356] "Generic (PLEG): container finished" podID="57701f93-f2e2-498d-93cf-f45099099aa2" containerID="bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1" exitCode=0
Jul 27 22:28:23.266050 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.266025    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"
Jul 27 22:28:23.266100 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.266024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" event={"ID":"57701f93-f2e2-498d-93cf-f45099099aa2","Type":"ContainerDied","Data":"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"}
Jul 27 22:28:23.266139 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.266125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h" event={"ID":"57701f93-f2e2-498d-93cf-f45099099aa2","Type":"ContainerDied","Data":"8ddebbdadbdb6d377970bc415a2561cef75217b3067d59358beda92fbca54db6"}
Jul 27 22:28:23.275107 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.275092    2578 scope.go:117] "RemoveContainer" containerID="4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"
Jul 27 22:28:23.275378 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:28:23.275359    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35\": container with ID starting with 4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35 not found: ID does not exist" containerID="4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"
Jul 27 22:28:23.275428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.275392    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35"} err="failed to get container status \"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35\": rpc error: code = NotFound desc = could not find container \"4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35\": container with ID starting with 4e3accf2fd4b73d99d99a74328449c8c12f2de788609efa3c16b6987a9612c35 not found: ID does not exist"
Jul 27 22:28:23.275428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.275412    2578 scope.go:117] "RemoveContainer" containerID="bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"
Jul 27 22:28:23.283295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.283276    2578 scope.go:117] "RemoveContainer" containerID="bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"
Jul 27 22:28:23.283615 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:28:23.283585    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1\": container with ID starting with bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1 not found: ID does not exist" containerID="bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"
Jul 27 22:28:23.283721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.283624    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1"} err="failed to get container status \"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1\": rpc error: code = NotFound desc = could not find container \"bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1\": container with ID starting with bde103cbb13953857e09a10ab3dc00813f86b47122d15ab663d61ee2b94021f1 not found: ID does not exist"
Jul 27 22:28:23.283721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.283640    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tkcj\" (UniqueName: \"kubernetes.io/projected/57701f93-f2e2-498d-93cf-f45099099aa2-kube-api-access-4tkcj\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:23.283721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.283664    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gskps\" (UniqueName: \"kubernetes.io/projected/00965727-aa42-4945-b20e-c96f1262a5df-kube-api-access-gskps\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:23.283721 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.283697    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00965727-aa42-4945-b20e-c96f1262a5df-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:23.289819 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.289788    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:28:23.293917 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.293895    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d2650623-68748bfdb8-8kd8h"]
Jul 27 22:28:23.304425 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.304403    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:28:23.308644 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:23.308619    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d2650623-cb5f84478-7nvtp"]
Jul 27 22:28:24.322745 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:24.322715    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00965727-aa42-4945-b20e-c96f1262a5df" path="/var/lib/kubelet/pods/00965727-aa42-4945-b20e-c96f1262a5df/volumes"
Jul 27 22:28:24.323147 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:24.323095    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57701f93-f2e2-498d-93cf-f45099099aa2" path="/var/lib/kubelet/pods/57701f93-f2e2-498d-93cf-f45099099aa2/volumes"
Jul 27 22:28:30.346773 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:30.346740    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:28:35.400849 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:35.400814    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:28:50.420379 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.420325    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" podUID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" containerName="main" containerID="cri-o://17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a" gracePeriod=45
Jul 27 22:28:50.511559 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.511523    2578 scope.go:117] "RemoveContainer" containerID="5052e66e97cb6a10a0bf2c4f14a6c625c69d88a0bc3565d1f5433c72ef698f33"
Jul 27 22:28:50.687270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.687246    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:28:50.731170 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731229    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731268    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731321    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5s9f\" (UniqueName: \"kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f\") pod \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\" (UID: \"e05be0af-4e32-48fb-9e37-71f935e7f2fa\") "
Jul 27 22:28:50.731347 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731342    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:50.731713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731452    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home" (OuterVolumeSpecName: "home") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:50.731713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731492    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:50.731713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731576    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:50.731713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731593    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:50.731713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.731604    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:50.733950 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.733922    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f" (OuterVolumeSpecName: "kube-api-access-f5s9f") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "kube-api-access-f5s9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:28:50.734075 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.734040    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm" (OuterVolumeSpecName: "dshm") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:28:50.734075 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.734053    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e05be0af-4e32-48fb-9e37-71f935e7f2fa" (UID: "e05be0af-4e32-48fb-9e37-71f935e7f2fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:28:50.833011 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.832982    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e05be0af-4e32-48fb-9e37-71f935e7f2fa-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:50.833011 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.833009    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e05be0af-4e32-48fb-9e37-71f935e7f2fa-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:50.833186 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:50.833021    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5s9f\" (UniqueName: \"kubernetes.io/projected/e05be0af-4e32-48fb-9e37-71f935e7f2fa-kube-api-access-f5s9f\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:28:51.413395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.413358    2578 generic.go:356] "Generic (PLEG): container finished" podID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" containerID="17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a" exitCode=0
Jul 27 22:28:51.413605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.413407    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"
Jul 27 22:28:51.413605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.413442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" event={"ID":"e05be0af-4e32-48fb-9e37-71f935e7f2fa","Type":"ContainerDied","Data":"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"}
Jul 27 22:28:51.413605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.413490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn" event={"ID":"e05be0af-4e32-48fb-9e37-71f935e7f2fa","Type":"ContainerDied","Data":"05c2ed9d23af3971ddc7864ca1264748ae65aaa55760647d048a8383d87c9cc7"}
Jul 27 22:28:51.413605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.413514    2578 scope.go:117] "RemoveContainer" containerID="17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"
Jul 27 22:28:51.423362 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.423164    2578 scope.go:117] "RemoveContainer" containerID="17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"
Jul 27 22:28:51.423598 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:28:51.423444    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a\": container with ID starting with 17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a not found: ID does not exist" containerID="17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"
Jul 27 22:28:51.423598 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.423468    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a"} err="failed to get container status \"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a\": rpc error: code = NotFound desc = could not find container \"17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a\": container with ID starting with 17a00ccb624dca60e1c68fc0300d032e8acd3490988e1cead3e50a2f59193e7a not found: ID does not exist"
Jul 27 22:28:51.439267 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.439247    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:28:51.443980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:51.443961    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d2650623/e2e-lifecycle-model-d2650623-kserve-8c6665779-nlfsn"]
Jul 27 22:28:52.322782 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:28:52.322749    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" path="/var/lib/kubelet/pods/e05be0af-4e32-48fb-9e37-71f935e7f2fa/volumes"
Jul 27 22:29:22.350712 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:29:22.350612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:29:33.936610 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:29:33.936575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:29:50.326538 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:29:50.326505    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:29:50.332186 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:29:50.332167    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:30:17.946214 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946177    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946547    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00965727-aa42-4945-b20e-c96f1262a5df" containerName="payload-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946557    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="00965727-aa42-4945-b20e-c96f1262a5df" containerName="payload-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946580    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57701f93-f2e2-498d-93cf-f45099099aa2" containerName="payload-pre-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946586    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="57701f93-f2e2-498d-93cf-f45099099aa2" containerName="payload-pre-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946598    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" containerName="main"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946603    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" containerName="main"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946656    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="57701f93-f2e2-498d-93cf-f45099099aa2" containerName="payload-pre-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946664    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="00965727-aa42-4945-b20e-c96f1262a5df" containerName="payload-processing"
Jul 27 22:30:17.946696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.946689    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05be0af-4e32-48fb-9e37-71f935e7f2fa" containerName="main"
Jul 27 22:30:17.949739 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.949723    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:17.952315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.952294    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d5bbdd\""
Jul 27 22:30:17.960422 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.960402    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:30:17.969488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.969467    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:30:17.972877 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.972856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:17.975477 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.975432    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d5bbdd-dockercfg-xzj8g\""
Jul 27 22:30:17.989887 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:17.989857    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:30:18.097244 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.097207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkv8m\" (UniqueName: \"kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.097443 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.097270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.097443 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.097332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.097443 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.097397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frx5\" (UniqueName: \"kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.198761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.198661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.198761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.198722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.198761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.198753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frx5\" (UniqueName: \"kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.199034 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.198800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkv8m\" (UniqueName: \"kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.199412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.199390    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.199412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.199402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.207796 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.207770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frx5\" (UniqueName: \"kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5\") pod \"payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.207910 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.207869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkv8m\" (UniqueName: \"kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m\") pod \"payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.260299 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.260271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.290128 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.290098    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.617033 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.617003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:30:18.619523 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:30:18.619490    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ed5671_82a2_4c74_be90_798c149c7dee.slice/crio-413918b0f0a835c02639b4f4af8caa48d4760761e7f9040e428998a9ec608068 WatchSource:0}: Error finding container 413918b0f0a835c02639b4f4af8caa48d4760761e7f9040e428998a9ec608068: Status 404 returned error can't find the container with id 413918b0f0a835c02639b4f4af8caa48d4760761e7f9040e428998a9ec608068
Jul 27 22:30:18.649965 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.649862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:30:18.655817 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:30:18.655789    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cb6e6f_947d_4d87_a69f_59ee798cebf1.slice/crio-dc9cbc2847dfe51bb13165ccff8229ace5efabbd8f01f3577ceff51cc76c21cb WatchSource:0}: Error finding container dc9cbc2847dfe51bb13165ccff8229ace5efabbd8f01f3577ceff51cc76c21cb: Status 404 returned error can't find the container with id dc9cbc2847dfe51bb13165ccff8229ace5efabbd8f01f3577ceff51cc76c21cb
Jul 27 22:30:18.846366 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.846326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" event={"ID":"98ed5671-82a2-4c74-be90-798c149c7dee","Type":"ContainerStarted","Data":"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"}
Jul 27 22:30:18.846545 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.846380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" event={"ID":"98ed5671-82a2-4c74-be90-798c149c7dee","Type":"ContainerStarted","Data":"413918b0f0a835c02639b4f4af8caa48d4760761e7f9040e428998a9ec608068"}
Jul 27 22:30:18.846545 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.846507    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:18.848032 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.848008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" event={"ID":"b5cb6e6f-947d-4d87-a69f-59ee798cebf1","Type":"ContainerStarted","Data":"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"}
Jul 27 22:30:18.848149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.848045    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" event={"ID":"b5cb6e6f-947d-4d87-a69f-59ee798cebf1","Type":"ContainerStarted","Data":"dc9cbc2847dfe51bb13165ccff8229ace5efabbd8f01f3577ceff51cc76c21cb"}
Jul 27 22:30:18.848149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.848135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:18.862150 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.862107    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" podStartSLOduration=1.8620937629999998 podStartE2EDuration="1.862093763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:30:18.860614525 +0000 UTC m=+2129.047455186" watchObservedRunningTime="2026-07-27 22:30:18.862093763 +0000 UTC m=+2129.048934421"
Jul 27 22:30:18.878467 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:18.878392    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" podStartSLOduration=1.878377497 podStartE2EDuration="1.878377497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:30:18.877121877 +0000 UTC m=+2129.063962537" watchObservedRunningTime="2026-07-27 22:30:18.878377497 +0000 UTC m=+2129.065218157"
Jul 27 22:30:24.039613 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:24.039578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:30:25.947160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.947128    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:30:25.950812 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.950794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:25.953434 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.953407    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-b2aa15\""
Jul 27 22:30:25.953584 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.953407    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-b2aa15-dockercfg-pwzsd\""
Jul 27 22:30:25.960693 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.960649    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:30:25.972338 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.972314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjggh\" (UniqueName: \"kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:25.972456 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.972344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:25.981814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.981793    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:30:25.985233 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.985213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:25.993712 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:25.993691    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:30:26.073039 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.073011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhr85\" (UniqueName: \"kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.073215 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.073048    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.073215 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.073117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjggh\" (UniqueName: \"kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.073215 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.073145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.073783 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.073767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.082385 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.082364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjggh\" (UniqueName: \"kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh\") pod \"payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.173817 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.173785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhr85\" (UniqueName: \"kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.173950 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.173832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.174537 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.174518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.182846 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.182816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhr85\" (UniqueName: \"kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85\") pod \"payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.262344 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.262316    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.297534 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.297499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.426488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.426464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:30:26.429195 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:30:26.429164    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd9a129_4669_43a2_acea_78582bc8b27e.slice/crio-2ada064db27b3bc3d4a6439a2c1b62ce8c8562d43dcc305d46f921448e5f3b31 WatchSource:0}: Error finding container 2ada064db27b3bc3d4a6439a2c1b62ce8c8562d43dcc305d46f921448e5f3b31: Status 404 returned error can't find the container with id 2ada064db27b3bc3d4a6439a2c1b62ce8c8562d43dcc305d46f921448e5f3b31
Jul 27 22:30:26.454490 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.454471    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:30:26.467742 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:30:26.467719    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9153aee_2b77_4faa_a891_e38b069b31a8.slice/crio-027b6c67839f29b7193bcbb8b03f2ecd4a9c7b933ed1f26a92435bc7937b5f35 WatchSource:0}: Error finding container 027b6c67839f29b7193bcbb8b03f2ecd4a9c7b933ed1f26a92435bc7937b5f35: Status 404 returned error can't find the container with id 027b6c67839f29b7193bcbb8b03f2ecd4a9c7b933ed1f26a92435bc7937b5f35
Jul 27 22:30:26.891605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.891513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" event={"ID":"c9153aee-2b77-4faa-a891-e38b069b31a8","Type":"ContainerStarted","Data":"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"}
Jul 27 22:30:26.891605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.891579    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:26.891905 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.891610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" event={"ID":"c9153aee-2b77-4faa-a891-e38b069b31a8","Type":"ContainerStarted","Data":"027b6c67839f29b7193bcbb8b03f2ecd4a9c7b933ed1f26a92435bc7937b5f35"}
Jul 27 22:30:26.893295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.893264    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" event={"ID":"bfd9a129-4669-43a2-acea-78582bc8b27e","Type":"ContainerStarted","Data":"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"}
Jul 27 22:30:26.893454 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.893310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" event={"ID":"bfd9a129-4669-43a2-acea-78582bc8b27e","Type":"ContainerStarted","Data":"2ada064db27b3bc3d4a6439a2c1b62ce8c8562d43dcc305d46f921448e5f3b31"}
Jul 27 22:30:26.893454 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.893330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:26.907210 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.907154    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" podStartSLOduration=1.907139231 podStartE2EDuration="1.907139231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:30:26.906582753 +0000 UTC m=+2137.093423412" watchObservedRunningTime="2026-07-27 22:30:26.907139231 +0000 UTC m=+2137.093979868"
Jul 27 22:30:26.924353 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:26.924297    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" podStartSLOduration=1.924280907 podStartE2EDuration="1.924280907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:30:26.922062535 +0000 UTC m=+2137.108903195" watchObservedRunningTime="2026-07-27 22:30:26.924280907 +0000 UTC m=+2137.111121568"
Jul 27 22:30:29.854125 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:29.854094    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:30:29.854530 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:29.854269    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:30:35.329372 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.329338    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:30:35.333019 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.332999    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.336841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.336809    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d5bbdd\"/\"kube-root-ca.crt\""
Jul 27 22:30:35.336984 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.336837    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d5bbdd\"/\"default-dockercfg-bckhs\""
Jul 27 22:30:35.336984 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.336863    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d5bbdd\"/\"openshift-service-ca.crt\""
Jul 27 22:30:35.336984 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.336815    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d5bbdd\"/\"test-model-d5bbdd-kserve-self-signed-certs\""
Jul 27 22:30:35.341726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.341703    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:30:35.460964 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.460930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.461141 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.460983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-str8b\" (UniqueName: \"kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.461141 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.461055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.461141 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.461116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.461261 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.461163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.461261 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.461179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-str8b\" (UniqueName: \"kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562650 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.562951 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.563019 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.563019 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.562999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.565160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.565136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.565348 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.565332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.571459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.571437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-str8b\" (UniqueName: \"kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b\") pod \"test-model-d5bbdd-kserve-74dc7f4b87-v8tfz\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") " pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.643906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.643832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:35.783506 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.783477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:30:35.787710 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:30:35.785626    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb14c1a57_5958_4d53_a13f_bcd2c852d94a.slice/crio-730e2064298bbcd716ae0c859e9ac7c2ca8033fe2d3af5b33bebbad692948bdc WatchSource:0}: Error finding container 730e2064298bbcd716ae0c859e9ac7c2ca8033fe2d3af5b33bebbad692948bdc: Status 404 returned error can't find the container with id 730e2064298bbcd716ae0c859e9ac7c2ca8033fe2d3af5b33bebbad692948bdc
Jul 27 22:30:35.855770 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.855736    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:30:35.942051 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.941964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" event={"ID":"b14c1a57-5958-4d53-a13f-bcd2c852d94a","Type":"ContainerStarted","Data":"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"}
Jul 27 22:30:35.942051 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.942003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" event={"ID":"b14c1a57-5958-4d53-a13f-bcd2c852d94a","Type":"ContainerStarted","Data":"730e2064298bbcd716ae0c859e9ac7c2ca8033fe2d3af5b33bebbad692948bdc"}
Jul 27 22:30:35.961742 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:35.961695    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" podStartSLOduration=0.961656313 podStartE2EDuration="961.656313ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:30:35.960329323 +0000 UTC m=+2146.147169984" watchObservedRunningTime="2026-07-27 22:30:35.961656313 +0000 UTC m=+2146.148496972"
Jul 27 22:30:37.899360 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:37.899330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:30:37.899828 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:37.899473    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:30:45.551353 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:45.551270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:30:45.644886 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:45.644852    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:45.645060 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:45.644905    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:45.657793 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:45.657764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:30:46.011225 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:30:46.011195    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:31:07.037842 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:07.037806    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:31:21.042966 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:21.042932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:31:43.859307 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:43.859273    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:31:48.551038 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:48.551002    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:31:55.463570 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.463537    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:31:55.464084 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.463770    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" podUID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" containerName="payload-processing" containerID="cri-o://a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339" gracePeriod=30
Jul 27 22:31:55.470426 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.470390    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:31:55.470605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.470576    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" podUID="98ed5671-82a2-4c74-be90-798c149c7dee" containerName="payload-pre-processing" containerID="cri-o://5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33" gracePeriod=30
Jul 27 22:31:55.754403 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.754380    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:31:55.757276 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.757257    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:31:55.788586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.787619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume\") pod \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") "
Jul 27 22:31:55.788586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.787694    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume\") pod \"98ed5671-82a2-4c74-be90-798c149c7dee\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") "
Jul 27 22:31:55.788586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.787738    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2frx5\" (UniqueName: \"kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5\") pod \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\" (UID: \"b5cb6e6f-947d-4d87-a69f-59ee798cebf1\") "
Jul 27 22:31:55.788586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.787766    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkv8m\" (UniqueName: \"kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m\") pod \"98ed5671-82a2-4c74-be90-798c149c7dee\" (UID: \"98ed5671-82a2-4c74-be90-798c149c7dee\") "
Jul 27 22:31:55.790035 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.789979    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5cb6e6f-947d-4d87-a69f-59ee798cebf1" (UID: "b5cb6e6f-947d-4d87-a69f-59ee798cebf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:31:55.790035 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.789982    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98ed5671-82a2-4c74-be90-798c149c7dee" (UID: "98ed5671-82a2-4c74-be90-798c149c7dee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:31:55.791850 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.791820    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5" (OuterVolumeSpecName: "kube-api-access-2frx5") pod "b5cb6e6f-947d-4d87-a69f-59ee798cebf1" (UID: "b5cb6e6f-947d-4d87-a69f-59ee798cebf1"). InnerVolumeSpecName "kube-api-access-2frx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:31:55.792175 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.792153    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m" (OuterVolumeSpecName: "kube-api-access-kkv8m") pod "98ed5671-82a2-4c74-be90-798c149c7dee" (UID: "98ed5671-82a2-4c74-be90-798c149c7dee"). InnerVolumeSpecName "kube-api-access-kkv8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:31:55.896424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.893633    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:55.896424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.893690    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed5671-82a2-4c74-be90-798c149c7dee-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:55.896424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.893706    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2frx5\" (UniqueName: \"kubernetes.io/projected/b5cb6e6f-947d-4d87-a69f-59ee798cebf1-kube-api-access-2frx5\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:55.896424 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:55.893719    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkv8m\" (UniqueName: \"kubernetes.io/projected/98ed5671-82a2-4c74-be90-798c149c7dee-kube-api-access-kkv8m\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:56.365760 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.365727    2578 generic.go:356] "Generic (PLEG): container finished" podID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" containerID="a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339" exitCode=0
Jul 27 22:31:56.365892 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.365790    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"
Jul 27 22:31:56.365892 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.365818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" event={"ID":"b5cb6e6f-947d-4d87-a69f-59ee798cebf1","Type":"ContainerDied","Data":"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"}
Jul 27 22:31:56.365892 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.365872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk" event={"ID":"b5cb6e6f-947d-4d87-a69f-59ee798cebf1","Type":"ContainerDied","Data":"dc9cbc2847dfe51bb13165ccff8229ace5efabbd8f01f3577ceff51cc76c21cb"}
Jul 27 22:31:56.366072 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.365898    2578 scope.go:117] "RemoveContainer" containerID="a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"
Jul 27 22:31:56.367193 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.367164    2578 generic.go:356] "Generic (PLEG): container finished" podID="98ed5671-82a2-4c74-be90-798c149c7dee" containerID="5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33" exitCode=0
Jul 27 22:31:56.367311 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.367233    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"
Jul 27 22:31:56.367311 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.367273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" event={"ID":"98ed5671-82a2-4c74-be90-798c149c7dee","Type":"ContainerDied","Data":"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"}
Jul 27 22:31:56.367488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.367327    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s" event={"ID":"98ed5671-82a2-4c74-be90-798c149c7dee","Type":"ContainerDied","Data":"413918b0f0a835c02639b4f4af8caa48d4760761e7f9040e428998a9ec608068"}
Jul 27 22:31:56.375157 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.375135    2578 scope.go:117] "RemoveContainer" containerID="a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"
Jul 27 22:31:56.375391 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:31:56.375373    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339\": container with ID starting with a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339 not found: ID does not exist" containerID="a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"
Jul 27 22:31:56.375444 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.375399    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339"} err="failed to get container status \"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339\": rpc error: code = NotFound desc = could not find container \"a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339\": container with ID starting with a08eede9ad2745345b479d0b5f96adfd25593f8604fafcbccf2cb2a80ac4d339 not found: ID does not exist"
Jul 27 22:31:56.375444 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.375416    2578 scope.go:117] "RemoveContainer" containerID="5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"
Jul 27 22:31:56.384299 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.384277    2578 scope.go:117] "RemoveContainer" containerID="5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"
Jul 27 22:31:56.384600 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:31:56.384572    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33\": container with ID starting with 5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33 not found: ID does not exist" containerID="5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"
Jul 27 22:31:56.384700 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.384611    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33"} err="failed to get container status \"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33\": rpc error: code = NotFound desc = could not find container \"5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33\": container with ID starting with 5e647e17fdc68e107c2ba8b9afd083e47f4a3628d47809bd4b43fe45ae0b9a33 not found: ID does not exist"
Jul 27 22:31:56.386110 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.386089    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:31:56.389740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.389719    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d5bbdd-7cf6c5fb76-g4g4s"]
Jul 27 22:31:56.402255 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.402234    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:31:56.404700 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:56.404680    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d5bbdd-7d95ddcbdd-x45wk"]
Jul 27 22:31:58.322612 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:58.322577    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98ed5671-82a2-4c74-be90-798c149c7dee" path="/var/lib/kubelet/pods/98ed5671-82a2-4c74-be90-798c149c7dee/volumes"
Jul 27 22:31:58.323132 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:58.323101    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" path="/var/lib/kubelet/pods/b5cb6e6f-947d-4d87-a69f-59ee798cebf1/volumes"
Jul 27 22:31:58.879423 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:58.879383    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" podUID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" containerName="main" containerID="cri-o://d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f" gracePeriod=45
Jul 27 22:31:59.143948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.143885    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:31:59.223778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223754    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.223918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.223918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223828    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.223918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223869    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.223918 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223891    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-str8b\" (UniqueName: \"kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.224110 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.223928    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home\") pod \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\" (UID: \"b14c1a57-5958-4d53-a13f-bcd2c852d94a\") "
Jul 27 22:31:59.224110 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.224035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:31:59.224110 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.224078    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:31:59.224270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.224226    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.224270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.224238    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.224270 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.224261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home" (OuterVolumeSpecName: "home") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:31:59.226258 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.226230    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm" (OuterVolumeSpecName: "dshm") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:31:59.226258 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.226242    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b" (OuterVolumeSpecName: "kube-api-access-str8b") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "kube-api-access-str8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:31:59.226399 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.226284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b14c1a57-5958-4d53-a13f-bcd2c852d94a" (UID: "b14c1a57-5958-4d53-a13f-bcd2c852d94a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:31:59.325580 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.325546    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.325580 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.325580    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b14c1a57-5958-4d53-a13f-bcd2c852d94a-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.325971 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.325592    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-str8b\" (UniqueName: \"kubernetes.io/projected/b14c1a57-5958-4d53-a13f-bcd2c852d94a-kube-api-access-str8b\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.325971 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.325601    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b14c1a57-5958-4d53-a13f-bcd2c852d94a-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:31:59.386212 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.386184    2578 generic.go:356] "Generic (PLEG): container finished" podID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" containerID="d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f" exitCode=0
Jul 27 22:31:59.386359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.386245    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"
Jul 27 22:31:59.386359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.386257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" event={"ID":"b14c1a57-5958-4d53-a13f-bcd2c852d94a","Type":"ContainerDied","Data":"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"}
Jul 27 22:31:59.386359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.386321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz" event={"ID":"b14c1a57-5958-4d53-a13f-bcd2c852d94a","Type":"ContainerDied","Data":"730e2064298bbcd716ae0c859e9ac7c2ca8033fe2d3af5b33bebbad692948bdc"}
Jul 27 22:31:59.386359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.386343    2578 scope.go:117] "RemoveContainer" containerID="d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"
Jul 27 22:31:59.395761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.395732    2578 scope.go:117] "RemoveContainer" containerID="d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"
Jul 27 22:31:59.396010 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:31:59.395986    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f\": container with ID starting with d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f not found: ID does not exist" containerID="d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"
Jul 27 22:31:59.396087 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.396013    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f"} err="failed to get container status \"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f\": rpc error: code = NotFound desc = could not find container \"d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f\": container with ID starting with d92aa35d379d42e392015b4ee2de68895f6fce5c9596f252ca876510d7c22b1f not found: ID does not exist"
Jul 27 22:31:59.410225 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.410204    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:31:59.412085 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:31:59.412064    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d5bbdd/test-model-d5bbdd-kserve-74dc7f4b87-v8tfz"]
Jul 27 22:32:00.323328 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:00.323294    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" path="/var/lib/kubelet/pods/b14c1a57-5958-4d53-a13f-bcd2c852d94a/volumes"
Jul 27 22:32:03.939448 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:03.939412    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:32:07.157958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.157924    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"]
Jul 27 22:32:07.158308 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158295    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" containerName="main"
Jul 27 22:32:07.158308 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158306    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" containerName="main"
Jul 27 22:32:07.158395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158315    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" containerName="payload-processing"
Jul 27 22:32:07.158395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158323    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" containerName="payload-processing"
Jul 27 22:32:07.158395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158335    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ed5671-82a2-4c74-be90-798c149c7dee" containerName="payload-pre-processing"
Jul 27 22:32:07.158395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158340    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ed5671-82a2-4c74-be90-798c149c7dee" containerName="payload-pre-processing"
Jul 27 22:32:07.158518 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158410    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14c1a57-5958-4d53-a13f-bcd2c852d94a" containerName="main"
Jul 27 22:32:07.158518 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158420    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5cb6e6f-947d-4d87-a69f-59ee798cebf1" containerName="payload-processing"
Jul 27 22:32:07.158518 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.158427    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="98ed5671-82a2-4c74-be90-798c149c7dee" containerName="payload-pre-processing"
Jul 27 22:32:07.162739 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.162720    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:07.165450 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.165428    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:32:07.165585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.165427    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:32:07.170928 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.170902    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"]
Jul 27 22:32:07.293059 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.293024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw\" (UID: \"4acdd31c-061d-48be-a8c9-6fc74cc09ce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:07.393932 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.393901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw\" (UID: \"4acdd31c-061d-48be-a8c9-6fc74cc09ce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:07.394514 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.394496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw\" (UID: \"4acdd31c-061d-48be-a8c9-6fc74cc09ce6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:07.475105 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.475021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:07.616534 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:07.616506    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"]
Jul 27 22:32:07.618993 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:32:07.618966    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acdd31c_061d_48be_a8c9_6fc74cc09ce6.slice/crio-7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a WatchSource:0}: Error finding container 7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a: Status 404 returned error can't find the container with id 7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a
Jul 27 22:32:08.435546 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:08.435512    2578 generic.go:356] "Generic (PLEG): container finished" podID="4acdd31c-061d-48be-a8c9-6fc74cc09ce6" containerID="2d3af6cf95defdf87f3de6d19444ab14dd7ef293cab0285ee9db4d2c4f8f0c37" exitCode=0
Jul 27 22:32:08.435944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:08.435585    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw" event={"ID":"4acdd31c-061d-48be-a8c9-6fc74cc09ce6","Type":"ContainerDied","Data":"2d3af6cf95defdf87f3de6d19444ab14dd7ef293cab0285ee9db4d2c4f8f0c37"}
Jul 27 22:32:08.435944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:08.435642    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw" event={"ID":"4acdd31c-061d-48be-a8c9-6fc74cc09ce6","Type":"ContainerStarted","Data":"7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a"}
Jul 27 22:32:09.571910 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:09.571885    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:09.718613 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:09.718529    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca\") pod \"4acdd31c-061d-48be-a8c9-6fc74cc09ce6\" (UID: \"4acdd31c-061d-48be-a8c9-6fc74cc09ce6\") "
Jul 27 22:32:09.718896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:09.718873    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4acdd31c-061d-48be-a8c9-6fc74cc09ce6" (UID: "4acdd31c-061d-48be-a8c9-6fc74cc09ce6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:32:09.819997 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:09.819961    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4acdd31c-061d-48be-a8c9-6fc74cc09ce6-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:32:10.445943 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:10.445913    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"
Jul 27 22:32:10.446117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:10.445948    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw" event={"ID":"4acdd31c-061d-48be-a8c9-6fc74cc09ce6","Type":"ContainerDied","Data":"7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a"}
Jul 27 22:32:10.446117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:10.445999    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df83c8aedd5d90c5aa9b11e542353a868936e7c121934093b4bb72379dea15a"
Jul 27 22:32:17.666518 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.666433    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:32:17.667066 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.666701    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerName="payload-processing" containerID="cri-o://e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6" gracePeriod=30
Jul 27 22:32:17.672895 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.672870    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:32:17.673081 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.673058    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerName="payload-pre-processing" containerID="cri-o://7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7" gracePeriod=30
Jul 27 22:32:17.953268 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.953244    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:32:17.956310 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:17.956292    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:32:18.097281 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097243    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume\") pod \"c9153aee-2b77-4faa-a891-e38b069b31a8\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") "
Jul 27 22:32:18.097470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097311    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjggh\" (UniqueName: \"kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh\") pod \"bfd9a129-4669-43a2-acea-78582bc8b27e\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") "
Jul 27 22:32:18.097470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097338    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume\") pod \"bfd9a129-4669-43a2-acea-78582bc8b27e\" (UID: \"bfd9a129-4669-43a2-acea-78582bc8b27e\") "
Jul 27 22:32:18.097470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097443    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhr85\" (UniqueName: \"kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85\") pod \"c9153aee-2b77-4faa-a891-e38b069b31a8\" (UID: \"c9153aee-2b77-4faa-a891-e38b069b31a8\") "
Jul 27 22:32:18.097750 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097724    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bfd9a129-4669-43a2-acea-78582bc8b27e" (UID: "bfd9a129-4669-43a2-acea-78582bc8b27e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:32:18.097750 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9153aee-2b77-4faa-a891-e38b069b31a8" (UID: "c9153aee-2b77-4faa-a891-e38b069b31a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:32:18.097944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097852    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfd9a129-4669-43a2-acea-78582bc8b27e-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:32:18.097944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.097870    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9153aee-2b77-4faa-a891-e38b069b31a8-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:32:18.099637 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.099615    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh" (OuterVolumeSpecName: "kube-api-access-xjggh") pod "bfd9a129-4669-43a2-acea-78582bc8b27e" (UID: "bfd9a129-4669-43a2-acea-78582bc8b27e"). InnerVolumeSpecName "kube-api-access-xjggh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:32:18.100030 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.100010    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85" (OuterVolumeSpecName: "kube-api-access-fhr85") pod "c9153aee-2b77-4faa-a891-e38b069b31a8" (UID: "c9153aee-2b77-4faa-a891-e38b069b31a8"). InnerVolumeSpecName "kube-api-access-fhr85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:32:18.199271 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.199203    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjggh\" (UniqueName: \"kubernetes.io/projected/bfd9a129-4669-43a2-acea-78582bc8b27e-kube-api-access-xjggh\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:32:18.199271 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.199227    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhr85\" (UniqueName: \"kubernetes.io/projected/c9153aee-2b77-4faa-a891-e38b069b31a8-kube-api-access-fhr85\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:32:18.488493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.488457    2578 generic.go:356] "Generic (PLEG): container finished" podID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerID="7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7" exitCode=0
Jul 27 22:32:18.488662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.488503    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"
Jul 27 22:32:18.488662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.488541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" event={"ID":"c9153aee-2b77-4faa-a891-e38b069b31a8","Type":"ContainerDied","Data":"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"}
Jul 27 22:32:18.488662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.488591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" event={"ID":"c9153aee-2b77-4faa-a891-e38b069b31a8","Type":"ContainerDied","Data":"027b6c67839f29b7193bcbb8b03f2ecd4a9c7b933ed1f26a92435bc7937b5f35"}
Jul 27 22:32:18.488662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.488621    2578 scope.go:117] "RemoveContainer" containerID="7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"
Jul 27 22:32:18.490307 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.490284    2578 generic.go:356] "Generic (PLEG): container finished" podID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerID="e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6" exitCode=0
Jul 27 22:32:18.490411 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.490323    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"
Jul 27 22:32:18.490411 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.490347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" event={"ID":"bfd9a129-4669-43a2-acea-78582bc8b27e","Type":"ContainerDied","Data":"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"}
Jul 27 22:32:18.490411 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.490383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" event={"ID":"bfd9a129-4669-43a2-acea-78582bc8b27e","Type":"ContainerDied","Data":"2ada064db27b3bc3d4a6439a2c1b62ce8c8562d43dcc305d46f921448e5f3b31"}
Jul 27 22:32:18.499422 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.499394    2578 scope.go:117] "RemoveContainer" containerID="7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"
Jul 27 22:32:18.499700 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:32:18.499660    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7\": container with ID starting with 7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7 not found: ID does not exist" containerID="7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"
Jul 27 22:32:18.499766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.499710    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7"} err="failed to get container status \"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7\": rpc error: code = NotFound desc = could not find container \"7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7\": container with ID starting with 7418f607f275505d4c60f9c9c8600ed65f8c716e552888c45d69366ef9d08bc7 not found: ID does not exist"
Jul 27 22:32:18.499766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.499727    2578 scope.go:117] "RemoveContainer" containerID="e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"
Jul 27 22:32:18.508262 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.508240    2578 scope.go:117] "RemoveContainer" containerID="e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"
Jul 27 22:32:18.508476 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.508458    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:32:18.508529 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:32:18.508501    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6\": container with ID starting with e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6 not found: ID does not exist" containerID="e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"
Jul 27 22:32:18.508580 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.508522    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6"} err="failed to get container status \"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6\": rpc error: code = NotFound desc = could not find container \"e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6\": container with ID starting with e99c5b320699ee8434abebbe5d39044c7e06976af4f4253e35513b20b10d00d6 not found: ID does not exist"
Jul 27 22:32:18.512030 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.512010    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74"]
Jul 27 22:32:18.522436 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.522408    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:32:18.526246 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.526228    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc"]
Jul 27 22:32:18.898872 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.898776    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-b2aa15-8d65ff968-446rc" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.65:9004: i/o timeout"
Jul 27 22:32:18.898872 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:18.898802    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b2aa15-84fffb4796-9wh74" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.66:9004: i/o timeout"
Jul 27 22:32:20.328491 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:20.324933    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" path="/var/lib/kubelet/pods/bfd9a129-4669-43a2-acea-78582bc8b27e/volumes"
Jul 27 22:32:20.328491 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:20.325483    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" path="/var/lib/kubelet/pods/c9153aee-2b77-4faa-a891-e38b069b31a8/volumes"
Jul 27 22:32:20.847125 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:20.847094    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:32:22.247214 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:22.247172    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"]
Jul 27 22:32:22.249614 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:22.249584    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b2aa15-4fzsw"]
Jul 27 22:32:22.322542 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:22.322508    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4acdd31c-061d-48be-a8c9-6fc74cc09ce6" path="/var/lib/kubelet/pods/4acdd31c-061d-48be-a8c9-6fc74cc09ce6/volumes"
Jul 27 22:32:33.539310 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539263    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539639    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerName="payload-pre-processing"
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539649    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerName="payload-pre-processing"
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539662    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerName="payload-processing"
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539681    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerName="payload-processing"
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539695    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4acdd31c-061d-48be-a8c9-6fc74cc09ce6" containerName="revoke-keys"
Jul 27 22:32:33.539713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539701    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4acdd31c-061d-48be-a8c9-6fc74cc09ce6" containerName="revoke-keys"
Jul 27 22:32:33.539914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539755    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9153aee-2b77-4faa-a891-e38b069b31a8" containerName="payload-pre-processing"
Jul 27 22:32:33.539914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539766    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd9a129-4669-43a2-acea-78582bc8b27e" containerName="payload-processing"
Jul 27 22:32:33.539914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.539772    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4acdd31c-061d-48be-a8c9-6fc74cc09ce6" containerName="revoke-keys"
Jul 27 22:32:33.542603 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.542579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.546341 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.546323    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-35e234\""
Jul 27 22:32:33.546470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.546452    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-35e234-dockercfg-6hrln\""
Jul 27 22:32:33.560053 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.560028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:32:33.572039 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.572017    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:32:33.575215 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.575195    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.584399 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.584376    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:32:33.632726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.632694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrp2s\" (UniqueName: \"kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.632876 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.632733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.632876 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.632778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.632876 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.632801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49c2s\" (UniqueName: \"kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.733640 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.733606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.733640 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.733645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49c2s\" (UniqueName: \"kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.733868 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.733762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrp2s\" (UniqueName: \"kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.733868 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.733781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.734453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.734433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.734489 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.734463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.742651 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.742618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49c2s\" (UniqueName: \"kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s\") pod \"payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.742800 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.742744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrp2s\" (UniqueName: \"kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s\") pod \"payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") " pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:33.852853 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.852765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:33.886429 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:33.886399    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:34.007331 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.007262    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:32:34.010514 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:32:34.010481    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0bc6f7_695c_4b62_b91d_81e892ccac22.slice/crio-0426e29f4393613d88cd29f7ebfd84c30a2c6bf93c04b2491452f211bf8d0514 WatchSource:0}: Error finding container 0426e29f4393613d88cd29f7ebfd84c30a2c6bf93c04b2491452f211bf8d0514: Status 404 returned error can't find the container with id 0426e29f4393613d88cd29f7ebfd84c30a2c6bf93c04b2491452f211bf8d0514
Jul 27 22:32:34.049304 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.049278    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:32:34.055195 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:32:34.055152    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebe39af_9fec_424a_ad0b_6318178327d7.slice/crio-3897e254dc660539d230ffd026ae03e2c25cabaf1a5b7052394805d6270672e5 WatchSource:0}: Error finding container 3897e254dc660539d230ffd026ae03e2c25cabaf1a5b7052394805d6270672e5: Status 404 returned error can't find the container with id 3897e254dc660539d230ffd026ae03e2c25cabaf1a5b7052394805d6270672e5
Jul 27 22:32:34.573891 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.573854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" event={"ID":"9ebe39af-9fec-424a-ad0b-6318178327d7","Type":"ContainerStarted","Data":"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"}
Jul 27 22:32:34.574327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.573909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" event={"ID":"9ebe39af-9fec-424a-ad0b-6318178327d7","Type":"ContainerStarted","Data":"3897e254dc660539d230ffd026ae03e2c25cabaf1a5b7052394805d6270672e5"}
Jul 27 22:32:34.574327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.573976    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:34.575493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.575463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" event={"ID":"fa0bc6f7-695c-4b62-b91d-81e892ccac22","Type":"ContainerStarted","Data":"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"}
Jul 27 22:32:34.575632 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.575507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" event={"ID":"fa0bc6f7-695c-4b62-b91d-81e892ccac22","Type":"ContainerStarted","Data":"0426e29f4393613d88cd29f7ebfd84c30a2c6bf93c04b2491452f211bf8d0514"}
Jul 27 22:32:34.575632 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.575550    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:34.591740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.591697    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" podStartSLOduration=1.591664245 podStartE2EDuration="1.591664245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:32:34.589301031 +0000 UTC m=+2264.776141691" watchObservedRunningTime="2026-07-27 22:32:34.591664245 +0000 UTC m=+2264.778504903"
Jul 27 22:32:34.607705 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:34.607641    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" podStartSLOduration=1.607622526 podStartE2EDuration="1.607622526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:32:34.605956658 +0000 UTC m=+2264.792797318" watchObservedRunningTime="2026-07-27 22:32:34.607622526 +0000 UTC m=+2264.794463190"
Jul 27 22:32:39.748190 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:39.748153    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:32:41.631775 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.631743    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:32:41.636180 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.636163    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.639151 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.639123    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-04ee43\""
Jul 27 22:32:41.639249 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.639160    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-04ee43-dockercfg-5r6dc\""
Jul 27 22:32:41.645413 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.645390    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:32:41.650960 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.650927    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:32:41.654239 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.654220    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.666402 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.666381    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:32:41.705645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.705620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzv9\" (UniqueName: \"kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.705778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.705718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.705778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.705753    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.705778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.705774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrl2k\" (UniqueName: \"kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.806663 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.806628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.806858 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.806708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.806858 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.806748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrl2k\" (UniqueName: \"kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.806858 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.806798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzv9\" (UniqueName: \"kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.807352 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.807334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.807430 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.807412    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.815026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.815000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrl2k\" (UniqueName: \"kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k\") pod \"payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") " pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:41.815201 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.815183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzv9\" (UniqueName: \"kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9\") pod \"payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.948112 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.948041    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:41.964295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:41.964271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:42.308983 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.308955    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:32:42.311253 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:32:42.311214    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc2a607_10c0_4c14_95e6_b3259a755817.slice/crio-1b92b04f2b83aff096734185b49ed9a657bbbc9e2723c1c9de7de59929e53504 WatchSource:0}: Error finding container 1b92b04f2b83aff096734185b49ed9a657bbbc9e2723c1c9de7de59929e53504: Status 404 returned error can't find the container with id 1b92b04f2b83aff096734185b49ed9a657bbbc9e2723c1c9de7de59929e53504
Jul 27 22:32:42.335022 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.334997    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:32:42.344740 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:32:42.344705    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b395df_6aa2_42a7_8eff_d1d6d0784673.slice/crio-898ae746cc790b66a97732447d1bdd65e71c7094b64932613f845071daf85477 WatchSource:0}: Error finding container 898ae746cc790b66a97732447d1bdd65e71c7094b64932613f845071daf85477: Status 404 returned error can't find the container with id 898ae746cc790b66a97732447d1bdd65e71c7094b64932613f845071daf85477
Jul 27 22:32:42.619848 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.619747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" event={"ID":"8bc2a607-10c0-4c14-95e6-b3259a755817","Type":"ContainerStarted","Data":"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"}
Jul 27 22:32:42.619848 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.619809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" event={"ID":"8bc2a607-10c0-4c14-95e6-b3259a755817","Type":"ContainerStarted","Data":"1b92b04f2b83aff096734185b49ed9a657bbbc9e2723c1c9de7de59929e53504"}
Jul 27 22:32:42.619848 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.619837    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:42.621448 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.621415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" event={"ID":"e1b395df-6aa2-42a7-8eff-d1d6d0784673","Type":"ContainerStarted","Data":"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"}
Jul 27 22:32:42.621594 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.621463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" event={"ID":"e1b395df-6aa2-42a7-8eff-d1d6d0784673","Type":"ContainerStarted","Data":"898ae746cc790b66a97732447d1bdd65e71c7094b64932613f845071daf85477"}
Jul 27 22:32:42.621594 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.621538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:42.636765 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.636718    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" podStartSLOduration=1.63670665 podStartE2EDuration="1.63670665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:32:42.635172694 +0000 UTC m=+2272.822013366" watchObservedRunningTime="2026-07-27 22:32:42.63670665 +0000 UTC m=+2272.823547308"
Jul 27 22:32:42.653111 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:42.653071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" podStartSLOduration=1.65305387 podStartE2EDuration="1.65305387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:32:42.651542683 +0000 UTC m=+2272.838383353" watchObservedRunningTime="2026-07-27 22:32:42.65305387 +0000 UTC m=+2272.839894530"
Jul 27 22:32:45.581902 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:45.581865    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.69:9004: connect: connection refused"
Jul 27 22:32:45.582278 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:45.582222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:32:53.628186 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:53.628157    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:32:53.628703 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:53.628604    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:32:55.581316 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:55.581286    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:32:55.938140 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:32:55.938057    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:33:08.915012 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:08.914981    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:33:08.915448 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:08.915190    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" podUID="9ebe39af-9fec-424a-ad0b-6318178327d7" containerName="payload-processing" containerID="cri-o://9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085" gracePeriod=30
Jul 27 22:33:08.922370 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:08.922337    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:33:08.922570 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:08.922545    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerName="payload-pre-processing" containerID="cri-o://61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235" gracePeriod=30
Jul 27 22:33:09.186167 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.186141    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:33:09.201137 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.201116    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:33:09.354630 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.354600    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume\") pod \"9ebe39af-9fec-424a-ad0b-6318178327d7\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") "
Jul 27 22:33:09.354839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.354659    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrp2s\" (UniqueName: \"kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s\") pod \"9ebe39af-9fec-424a-ad0b-6318178327d7\" (UID: \"9ebe39af-9fec-424a-ad0b-6318178327d7\") "
Jul 27 22:33:09.354839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.354710    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49c2s\" (UniqueName: \"kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s\") pod \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") "
Jul 27 22:33:09.354839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.354799    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume\") pod \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\" (UID: \"fa0bc6f7-695c-4b62-b91d-81e892ccac22\") "
Jul 27 22:33:09.355133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.355108    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ebe39af-9fec-424a-ad0b-6318178327d7" (UID: "9ebe39af-9fec-424a-ad0b-6318178327d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:33:09.355204 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.355152    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa0bc6f7-695c-4b62-b91d-81e892ccac22" (UID: "fa0bc6f7-695c-4b62-b91d-81e892ccac22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:33:09.356931 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.356909    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s" (OuterVolumeSpecName: "kube-api-access-mrp2s") pod "9ebe39af-9fec-424a-ad0b-6318178327d7" (UID: "9ebe39af-9fec-424a-ad0b-6318178327d7"). InnerVolumeSpecName "kube-api-access-mrp2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:33:09.356931 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.356919    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s" (OuterVolumeSpecName: "kube-api-access-49c2s") pod "fa0bc6f7-695c-4b62-b91d-81e892ccac22" (UID: "fa0bc6f7-695c-4b62-b91d-81e892ccac22"). InnerVolumeSpecName "kube-api-access-49c2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:33:09.456248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.456184    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ebe39af-9fec-424a-ad0b-6318178327d7-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:09.456248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.456208    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrp2s\" (UniqueName: \"kubernetes.io/projected/9ebe39af-9fec-424a-ad0b-6318178327d7-kube-api-access-mrp2s\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:09.456248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.456218    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49c2s\" (UniqueName: \"kubernetes.io/projected/fa0bc6f7-695c-4b62-b91d-81e892ccac22-kube-api-access-49c2s\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:09.456248 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.456226    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa0bc6f7-695c-4b62-b91d-81e892ccac22-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:09.759192 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.759159    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ebe39af-9fec-424a-ad0b-6318178327d7" containerID="9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085" exitCode=0
Jul 27 22:33:09.759368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.759205    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"
Jul 27 22:33:09.759368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.759249    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" event={"ID":"9ebe39af-9fec-424a-ad0b-6318178327d7","Type":"ContainerDied","Data":"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"}
Jul 27 22:33:09.759368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.759306    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq" event={"ID":"9ebe39af-9fec-424a-ad0b-6318178327d7","Type":"ContainerDied","Data":"3897e254dc660539d230ffd026ae03e2c25cabaf1a5b7052394805d6270672e5"}
Jul 27 22:33:09.759368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.759336    2578 scope.go:117] "RemoveContainer" containerID="9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"
Jul 27 22:33:09.760581 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.760560    2578 generic.go:356] "Generic (PLEG): container finished" podID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerID="61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235" exitCode=0
Jul 27 22:33:09.760662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.760637    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"
Jul 27 22:33:09.760766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.760702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" event={"ID":"fa0bc6f7-695c-4b62-b91d-81e892ccac22","Type":"ContainerDied","Data":"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"}
Jul 27 22:33:09.760766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.760737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5" event={"ID":"fa0bc6f7-695c-4b62-b91d-81e892ccac22","Type":"ContainerDied","Data":"0426e29f4393613d88cd29f7ebfd84c30a2c6bf93c04b2491452f211bf8d0514"}
Jul 27 22:33:09.769897 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.769877    2578 scope.go:117] "RemoveContainer" containerID="9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"
Jul 27 22:33:09.770264 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:33:09.770138    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085\": container with ID starting with 9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085 not found: ID does not exist" containerID="9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"
Jul 27 22:33:09.770264 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.770163    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085"} err="failed to get container status \"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085\": rpc error: code = NotFound desc = could not find container \"9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085\": container with ID starting with 9daa1ea99a0bcadef218b6ecec386dc457fe9ce351bd3cde73bd679990d4b085 not found: ID does not exist"
Jul 27 22:33:09.770264 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.770180    2578 scope.go:117] "RemoveContainer" containerID="61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"
Jul 27 22:33:09.778315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.778297    2578 scope.go:117] "RemoveContainer" containerID="61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"
Jul 27 22:33:09.778620 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:33:09.778586    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235\": container with ID starting with 61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235 not found: ID does not exist" containerID="61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"
Jul 27 22:33:09.778747 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.778618    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235"} err="failed to get container status \"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235\": rpc error: code = NotFound desc = could not find container \"61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235\": container with ID starting with 61d2153b03a5cb3f4a9e12cfaf7b457c46fef40a58a5ff73b5ae71adda27c235 not found: ID does not exist"
Jul 27 22:33:09.780853 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.780834    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:33:09.786465 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.786442    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-35e234-7d98d6fbb4-vwwfq"]
Jul 27 22:33:09.795454 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.795434    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:33:09.798794 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:09.798774    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-35e234-78db444f44-dg9t5"]
Jul 27 22:33:10.322847 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:10.322818    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebe39af-9fec-424a-ad0b-6318178327d7" path="/var/lib/kubelet/pods/9ebe39af-9fec-424a-ad0b-6318178327d7/volumes"
Jul 27 22:33:10.323206 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:10.323176    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" path="/var/lib/kubelet/pods/fa0bc6f7-695c-4b62-b91d-81e892ccac22/volumes"
Jul 27 22:33:11.544472 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:11.544437    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:33:20.487404 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487366    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"]
Jul 27 22:33:20.487807 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487797    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerName="payload-pre-processing"
Jul 27 22:33:20.487851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487809    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerName="payload-pre-processing"
Jul 27 22:33:20.487851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487826    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebe39af-9fec-424a-ad0b-6318178327d7" containerName="payload-processing"
Jul 27 22:33:20.487851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487837    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebe39af-9fec-424a-ad0b-6318178327d7" containerName="payload-processing"
Jul 27 22:33:20.487954 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487902    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa0bc6f7-695c-4b62-b91d-81e892ccac22" containerName="payload-pre-processing"
Jul 27 22:33:20.487954 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.487916    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebe39af-9fec-424a-ad0b-6318178327d7" containerName="payload-processing"
Jul 27 22:33:20.492733 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.492714    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:20.495788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.495767    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:33:20.495927 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.495771    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:33:20.500662 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.500355    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"]
Jul 27 22:33:20.554083 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.554053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d\" (UID: \"bed2b4be-9699-4576-987f-8a9cdee8b740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:20.654777 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.654748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d\" (UID: \"bed2b4be-9699-4576-987f-8a9cdee8b740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:20.655344 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.655326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d\" (UID: \"bed2b4be-9699-4576-987f-8a9cdee8b740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:20.804458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.804384    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:20.941446 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:20.941423    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"]
Jul 27 22:33:20.943424 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:33:20.943393    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed2b4be_9699_4576_987f_8a9cdee8b740.slice/crio-cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c WatchSource:0}: Error finding container cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c: Status 404 returned error can't find the container with id cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c
Jul 27 22:33:21.827752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:21.827707    2578 generic.go:356] "Generic (PLEG): container finished" podID="bed2b4be-9699-4576-987f-8a9cdee8b740" containerID="e308efafbb7f901c150101fe4af682197d26125e011e567c8ad18d12f8bc3241" exitCode=0
Jul 27 22:33:21.828224 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:21.827770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d" event={"ID":"bed2b4be-9699-4576-987f-8a9cdee8b740","Type":"ContainerDied","Data":"e308efafbb7f901c150101fe4af682197d26125e011e567c8ad18d12f8bc3241"}
Jul 27 22:33:21.828224 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:21.827848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d" event={"ID":"bed2b4be-9699-4576-987f-8a9cdee8b740","Type":"ContainerStarted","Data":"cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c"}
Jul 27 22:33:22.972023 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:22.971995    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:23.072985 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.072949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca\") pod \"bed2b4be-9699-4576-987f-8a9cdee8b740\" (UID: \"bed2b4be-9699-4576-987f-8a9cdee8b740\") "
Jul 27 22:33:23.073304 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.073281    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bed2b4be-9699-4576-987f-8a9cdee8b740" (UID: "bed2b4be-9699-4576-987f-8a9cdee8b740"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:33:23.174537 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.174460    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed2b4be-9699-4576-987f-8a9cdee8b740-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:23.839711 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.839665    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"
Jul 27 22:33:23.839711 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.839695    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d" event={"ID":"bed2b4be-9699-4576-987f-8a9cdee8b740","Type":"ContainerDied","Data":"cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c"}
Jul 27 22:33:23.839913 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:23.839737    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf7bb918b7cdd9ed83e4240d3396a55d163f864c0949a2886417630cc1ae0c2c"
Jul 27 22:33:36.124715 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.124664    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:33:36.125195 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.124977    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" podUID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" containerName="payload-processing" containerID="cri-o://5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19" gracePeriod=30
Jul 27 22:33:36.133088 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.133051    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:33:36.133306 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.133278    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" podUID="8bc2a607-10c0-4c14-95e6-b3259a755817" containerName="payload-pre-processing" containerID="cri-o://9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190" gracePeriod=30
Jul 27 22:33:36.416054 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.416029    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:33:36.419032 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.419008    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:33:36.488011 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.487960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume\") pod \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") "
Jul 27 22:33:36.488011 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.488008    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrl2k\" (UniqueName: \"kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k\") pod \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\" (UID: \"e1b395df-6aa2-42a7-8eff-d1d6d0784673\") "
Jul 27 22:33:36.488266 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.488097    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhzv9\" (UniqueName: \"kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9\") pod \"8bc2a607-10c0-4c14-95e6-b3259a755817\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") "
Jul 27 22:33:36.488266 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.488156    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume\") pod \"8bc2a607-10c0-4c14-95e6-b3259a755817\" (UID: \"8bc2a607-10c0-4c14-95e6-b3259a755817\") "
Jul 27 22:33:36.488433 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.488400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1b395df-6aa2-42a7-8eff-d1d6d0784673" (UID: "e1b395df-6aa2-42a7-8eff-d1d6d0784673"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:33:36.488783 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.488758    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8bc2a607-10c0-4c14-95e6-b3259a755817" (UID: "8bc2a607-10c0-4c14-95e6-b3259a755817"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:33:36.490325 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.490303    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k" (OuterVolumeSpecName: "kube-api-access-mrl2k") pod "e1b395df-6aa2-42a7-8eff-d1d6d0784673" (UID: "e1b395df-6aa2-42a7-8eff-d1d6d0784673"). InnerVolumeSpecName "kube-api-access-mrl2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:33:36.491117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.491095    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9" (OuterVolumeSpecName: "kube-api-access-lhzv9") pod "8bc2a607-10c0-4c14-95e6-b3259a755817" (UID: "8bc2a607-10c0-4c14-95e6-b3259a755817"). InnerVolumeSpecName "kube-api-access-lhzv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:33:36.589158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.589115    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b395df-6aa2-42a7-8eff-d1d6d0784673-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:36.589158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.589157    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrl2k\" (UniqueName: \"kubernetes.io/projected/e1b395df-6aa2-42a7-8eff-d1d6d0784673-kube-api-access-mrl2k\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:36.589356 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.589172    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhzv9\" (UniqueName: \"kubernetes.io/projected/8bc2a607-10c0-4c14-95e6-b3259a755817-kube-api-access-lhzv9\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:36.589356 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.589186    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bc2a607-10c0-4c14-95e6-b3259a755817-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:33:36.906201 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.906161    2578 generic.go:356] "Generic (PLEG): container finished" podID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" containerID="5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19" exitCode=0
Jul 27 22:33:36.906359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.906209    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"
Jul 27 22:33:36.906359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.906242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" event={"ID":"e1b395df-6aa2-42a7-8eff-d1d6d0784673","Type":"ContainerDied","Data":"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"}
Jul 27 22:33:36.906359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.906299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb" event={"ID":"e1b395df-6aa2-42a7-8eff-d1d6d0784673","Type":"ContainerDied","Data":"898ae746cc790b66a97732447d1bdd65e71c7094b64932613f845071daf85477"}
Jul 27 22:33:36.906359 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.906321    2578 scope.go:117] "RemoveContainer" containerID="5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"
Jul 27 22:33:36.907706 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.907683    2578 generic.go:356] "Generic (PLEG): container finished" podID="8bc2a607-10c0-4c14-95e6-b3259a755817" containerID="9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190" exitCode=0
Jul 27 22:33:36.907940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.907716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" event={"ID":"8bc2a607-10c0-4c14-95e6-b3259a755817","Type":"ContainerDied","Data":"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"}
Jul 27 22:33:36.907940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.907772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx" event={"ID":"8bc2a607-10c0-4c14-95e6-b3259a755817","Type":"ContainerDied","Data":"1b92b04f2b83aff096734185b49ed9a657bbbc9e2723c1c9de7de59929e53504"}
Jul 27 22:33:36.907940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.907727    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"
Jul 27 22:33:36.924183 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.924164    2578 scope.go:117] "RemoveContainer" containerID="5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"
Jul 27 22:33:36.924489 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:33:36.924473    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19\": container with ID starting with 5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19 not found: ID does not exist" containerID="5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"
Jul 27 22:33:36.924532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.924495    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19"} err="failed to get container status \"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19\": rpc error: code = NotFound desc = could not find container \"5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19\": container with ID starting with 5d20853cb48de2fe206711b4a2362c78108487205e5c2ee4964a5b1229cf0d19 not found: ID does not exist"
Jul 27 22:33:36.924532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.924513    2578 scope.go:117] "RemoveContainer" containerID="9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"
Jul 27 22:33:36.932588 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.932569    2578 scope.go:117] "RemoveContainer" containerID="9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"
Jul 27 22:33:36.932870 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:33:36.932850    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190\": container with ID starting with 9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190 not found: ID does not exist" containerID="9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"
Jul 27 22:33:36.932942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.932878    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190"} err="failed to get container status \"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190\": rpc error: code = NotFound desc = could not find container \"9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190\": container with ID starting with 9f3a1a7c5b181ba280670cd6f9cc3366ae25a09449e6f39f2586cef5db83c190 not found: ID does not exist"
Jul 27 22:33:36.935855 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.935833    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:33:36.942862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.942842    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-04ee43-7858dc659b-q6tdx"]
Jul 27 22:33:36.955427 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.955402    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:33:36.959436 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:36.959412    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-04ee43-545fccf768-xsmkb"]
Jul 27 22:33:38.323303 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:38.323265    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc2a607-10c0-4c14-95e6-b3259a755817" path="/var/lib/kubelet/pods/8bc2a607-10c0-4c14-95e6-b3259a755817/volumes"
Jul 27 22:33:38.323706 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:38.323609    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" path="/var/lib/kubelet/pods/e1b395df-6aa2-42a7-8eff-d1d6d0784673/volumes"
Jul 27 22:33:40.567361 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:40.567325    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"]
Jul 27 22:33:40.570510 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:40.570483    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee43-hbn8d"]
Jul 27 22:33:40.844938 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:40.844862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:33:42.322794 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:42.322760    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bed2b4be-9699-4576-987f-8a9cdee8b740" path="/var/lib/kubelet/pods/bed2b4be-9699-4576-987f-8a9cdee8b740/volumes"
Jul 27 22:33:52.021553 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021522    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021906    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc2a607-10c0-4c14-95e6-b3259a755817" containerName="payload-pre-processing"
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021918    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc2a607-10c0-4c14-95e6-b3259a755817" containerName="payload-pre-processing"
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021945    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" containerName="payload-processing"
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021951    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" containerName="payload-processing"
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021958    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed2b4be-9699-4576-987f-8a9cdee8b740" containerName="revoke-keys"
Jul 27 22:33:52.021969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.021965    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed2b4be-9699-4576-987f-8a9cdee8b740" containerName="revoke-keys"
Jul 27 22:33:52.022173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.022022    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed2b4be-9699-4576-987f-8a9cdee8b740" containerName="revoke-keys"
Jul 27 22:33:52.022173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.022033    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b395df-6aa2-42a7-8eff-d1d6d0784673" containerName="payload-processing"
Jul 27 22:33:52.022173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.022041    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc2a607-10c0-4c14-95e6-b3259a755817" containerName="payload-pre-processing"
Jul 27 22:33:52.026839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.026817    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.029413 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.029391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-c5cbe3-dockercfg-rc8d7\""
Jul 27 22:33:52.029535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.029468    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-c5cbe3\""
Jul 27 22:33:52.036910 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.036888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:33:52.052613 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.052584    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:33:52.058101 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.057109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.066539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.066516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:33:52.130765 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.130732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rsfc\" (UniqueName: \"kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.130906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.130768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfx7x\" (UniqueName: \"kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.130906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.130800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.131062 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.130920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.231796 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.231760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.231940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.231849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.231940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.231898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rsfc\" (UniqueName: \"kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.231940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.231924    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfx7x\" (UniqueName: \"kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.232459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.232438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.232604 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.232585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.240779 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.240757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfx7x\" (UniqueName: \"kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x\") pod \"payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.240914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.240811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rsfc\" (UniqueName: \"kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc\") pod \"payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.338226 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.338153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.373063 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.373034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:52.492710 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.492648    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:33:52.500570 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:33:52.500526    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4dd926_2fb1_4839_8485_c9436d30aeba.slice/crio-f306c0c026eff98636d184cabbdc1f59ac998005158545f095cb331e8fa7e44b WatchSource:0}: Error finding container f306c0c026eff98636d184cabbdc1f59ac998005158545f095cb331e8fa7e44b: Status 404 returned error can't find the container with id f306c0c026eff98636d184cabbdc1f59ac998005158545f095cb331e8fa7e44b
Jul 27 22:33:52.530007 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.529976    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:33:52.535851 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:33:52.535822    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fbbfaa_5801_4ec5_b364_195b7698c1b4.slice/crio-31344f33669660d05f4e3176897a990cd22ba5237ced9e386b8a224b29a0f06d WatchSource:0}: Error finding container 31344f33669660d05f4e3176897a990cd22ba5237ced9e386b8a224b29a0f06d: Status 404 returned error can't find the container with id 31344f33669660d05f4e3176897a990cd22ba5237ced9e386b8a224b29a0f06d
Jul 27 22:33:52.993906 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.993878    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" event={"ID":"eb4dd926-2fb1-4839-8485-c9436d30aeba","Type":"ContainerStarted","Data":"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"}
Jul 27 22:33:52.994301 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.993935    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" event={"ID":"eb4dd926-2fb1-4839-8485-c9436d30aeba","Type":"ContainerStarted","Data":"f306c0c026eff98636d184cabbdc1f59ac998005158545f095cb331e8fa7e44b"}
Jul 27 22:33:52.994301 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.993967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:33:52.995921 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.995893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" event={"ID":"12fbbfaa-5801-4ec5-b364-195b7698c1b4","Type":"ContainerStarted","Data":"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"}
Jul 27 22:33:52.996042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.995933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" event={"ID":"12fbbfaa-5801-4ec5-b364-195b7698c1b4","Type":"ContainerStarted","Data":"31344f33669660d05f4e3176897a990cd22ba5237ced9e386b8a224b29a0f06d"}
Jul 27 22:33:52.996042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:52.995962    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:33:53.028221 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:53.028174    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" podStartSLOduration=1.028159787 podStartE2EDuration="1.028159787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:33:53.026834326 +0000 UTC m=+2343.213674985" watchObservedRunningTime="2026-07-27 22:33:53.028159787 +0000 UTC m=+2343.215000445"
Jul 27 22:33:53.053153 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:53.053108    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" podStartSLOduration=1.053093782 podStartE2EDuration="1.053093782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:33:53.051866249 +0000 UTC m=+2343.238706912" watchObservedRunningTime="2026-07-27 22:33:53.053093782 +0000 UTC m=+2343.239934440"
Jul 27 22:33:59.980051 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:59.980013    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:33:59.983740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:59.983712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:33:59.986950 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:59.986928    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-f36b41\""
Jul 27 22:33:59.987156 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:59.987029    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-f36b41-dockercfg-xj8pd\""
Jul 27 22:33:59.995472 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:33:59.995132    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:34:00.019753 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.019720    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:34:00.023311 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.023295    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.033133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.033104    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:34:00.059522 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.059481    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:34:00.102029 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.101996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.102029 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.102038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgwxd\" (UniqueName: \"kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.102252 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.102180    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.102252 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.102207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8fqf\" (UniqueName: \"kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.203097 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203060    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.203293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8fqf\" (UniqueName: \"kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.203293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.203293 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgwxd\" (UniqueName: \"kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.203896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.203981 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.203934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.216073 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.216048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgwxd\" (UniqueName: \"kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd\") pod \"payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.216177 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.216133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8fqf\" (UniqueName: \"kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf\") pod \"payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.298725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.298690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:00.335250 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.335216    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:00.470508 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.470476    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:34:00.475739 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:34:00.475707    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee62c1a_c753_459f_96a2_b06a04cb358d.slice/crio-8d0bcf5943c9ba0bc0f95435465397e40504c253ad1e156e3e63a843efec7d69 WatchSource:0}: Error finding container 8d0bcf5943c9ba0bc0f95435465397e40504c253ad1e156e3e63a843efec7d69: Status 404 returned error can't find the container with id 8d0bcf5943c9ba0bc0f95435465397e40504c253ad1e156e3e63a843efec7d69
Jul 27 22:34:00.516070 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:00.516044    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:34:00.525831 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:34:00.525801    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9348ed3_7f9c_4364_a307_2adcc09f135b.slice/crio-572cb60bf1022ef5c8c8f6e0a77253819346a2390482a64e301e3eb488d03ee2 WatchSource:0}: Error finding container 572cb60bf1022ef5c8c8f6e0a77253819346a2390482a64e301e3eb488d03ee2: Status 404 returned error can't find the container with id 572cb60bf1022ef5c8c8f6e0a77253819346a2390482a64e301e3eb488d03ee2
Jul 27 22:34:01.042043 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.042002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" event={"ID":"b9348ed3-7f9c-4364-a307-2adcc09f135b","Type":"ContainerStarted","Data":"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"}
Jul 27 22:34:01.042481 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.042077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" event={"ID":"b9348ed3-7f9c-4364-a307-2adcc09f135b","Type":"ContainerStarted","Data":"572cb60bf1022ef5c8c8f6e0a77253819346a2390482a64e301e3eb488d03ee2"}
Jul 27 22:34:01.042481 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.042204    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:01.043387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.043367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" event={"ID":"8ee62c1a-c753-459f-96a2-b06a04cb358d","Type":"ContainerStarted","Data":"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"}
Jul 27 22:34:01.043496 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.043399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" event={"ID":"8ee62c1a-c753-459f-96a2-b06a04cb358d","Type":"ContainerStarted","Data":"8d0bcf5943c9ba0bc0f95435465397e40504c253ad1e156e3e63a843efec7d69"}
Jul 27 22:34:01.043553 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.043511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:01.061485 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.061437    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" podStartSLOduration=2.061423146 podStartE2EDuration="2.061423146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:34:01.059455751 +0000 UTC m=+2351.246296414" watchObservedRunningTime="2026-07-27 22:34:01.061423146 +0000 UTC m=+2351.248263804"
Jul 27 22:34:01.078900 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:01.078850    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" podStartSLOduration=2.078832575 podStartE2EDuration="2.078832575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:34:01.078488543 +0000 UTC m=+2351.265329201" watchObservedRunningTime="2026-07-27 22:34:01.078832575 +0000 UTC m=+2351.265673233"
Jul 27 22:34:04.003579 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:04.003530    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.74:9004: connect: connection refused"
Jul 27 22:34:04.004038 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:04.003899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:34:11.449374 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:11.449339    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:34:12.050247 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:12.050213    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:34:12.050553 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:12.050534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:34:14.003092 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:14.003057    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:34:26.345118 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.345084    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:34:26.347616 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.347594    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.350350 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.350331    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"auth-test-model-c5cbe3-kserve-self-signed-certs\""
Jul 27 22:34:26.351470 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.351454    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"kube-root-ca.crt\""
Jul 27 22:34:26.351696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.351682    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"default-dockercfg-5dgpq\""
Jul 27 22:34:26.351782 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.351737    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"openshift-service-ca.crt\""
Jul 27 22:34:26.358331 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.358309    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:34:26.442439 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.442439 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.442725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.442725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.442725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.442725 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.442602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q688\" (UniqueName: \"kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543441 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q688\" (UniqueName: \"kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543937 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.543980 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.543973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.546042 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.546019    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.546315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.546295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.552189 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.552172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q688\" (UniqueName: \"kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688\") pod \"auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") " pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.657554 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.657464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:26.808211 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:26.808187    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:34:26.811088 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:34:26.811058    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b5ef4b7_074c_45a7_ac4f_a8cec48081f3.slice/crio-b38ac03569e4acb4b1c9feb661d3f419c8334e3c48c23db65585efd56fdb5aed WatchSource:0}: Error finding container b38ac03569e4acb4b1c9feb661d3f419c8334e3c48c23db65585efd56fdb5aed: Status 404 returned error can't find the container with id b38ac03569e4acb4b1c9feb661d3f419c8334e3c48c23db65585efd56fdb5aed
Jul 27 22:34:27.177015 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:27.176973    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" event={"ID":"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3","Type":"ContainerStarted","Data":"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"}
Jul 27 22:34:27.177212 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:27.177028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" event={"ID":"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3","Type":"ContainerStarted","Data":"b38ac03569e4acb4b1c9feb661d3f419c8334e3c48c23db65585efd56fdb5aed"}
Jul 27 22:34:27.199335 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:27.199273    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" podStartSLOduration=1.199257062 podStartE2EDuration="1.199257062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:34:27.196254077 +0000 UTC m=+2377.383094736" watchObservedRunningTime="2026-07-27 22:34:27.199257062 +0000 UTC m=+2377.386097728"
Jul 27 22:34:29.548113 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:29.548075    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:34:36.657663 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:36.657618    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:36.658092 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:36.657707    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:36.670392 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:36.670366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:37.242465 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:37.242432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:34:50.356939 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:50.356908    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:34:50.365683 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:50.365638    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:34:56.475455 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:34:56.475419    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:35:26.341748 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:35:26.341651    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:35:46.939098 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:35:46.939065    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:36:06.739656 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:36:06.739618    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:36:28.253484 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:36:28.253448    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:36:45.451876 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:36:45.451797    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:37:08.842962 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:37:08.842926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:37:25.540921 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:37:25.540885    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:37:52.244583 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:37:52.244550    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:38:10.655171 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:38:10.655134    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:38:37.543762 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:38:37.543657    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:38:50.663798 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:38:50.663768    2578 scope.go:117] "RemoveContainer" containerID="2d3af6cf95defdf87f3de6d19444ab14dd7ef293cab0285ee9db4d2c4f8f0c37"
Jul 27 22:38:54.642963 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:38:54.642923    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:39:16.340602 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:16.340571    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:39:20.395628 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:20.395593    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:39:31.644577 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.644543    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:39:31.648834 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.648803    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.651606 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.651582    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"sub-test-model-c5cbe3-kserve-self-signed-certs\""
Jul 27 22:39:31.658088 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.658059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:39:31.803849 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.803816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.804020 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.803856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.804020 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.803917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.804020 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.803995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgf26\" (UniqueName: \"kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.804178 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.804027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.804178 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.804054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.904786 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgf26\" (UniqueName: \"kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.904786 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.904786 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.904872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905217 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.905181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905333 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.905189    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.905333 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.905267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.907214 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.907190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.907451 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.907432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.913239 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.913214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgf26\" (UniqueName: \"kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26\") pod \"sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") " pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:31.959439 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:31.959406    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:32.098841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:32.098810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:39:32.100720 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:39:32.100696    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf2ab7ea_9ee3_4235_88dd_8bc32c406bd7.slice/crio-157251edd5a3c27a1136b7120b1eba5addf8669785be7467930ac9d4ffec937f WatchSource:0}: Error finding container 157251edd5a3c27a1136b7120b1eba5addf8669785be7467930ac9d4ffec937f: Status 404 returned error can't find the container with id 157251edd5a3c27a1136b7120b1eba5addf8669785be7467930ac9d4ffec937f
Jul 27 22:39:32.771077 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:32.771047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" event={"ID":"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7","Type":"ContainerStarted","Data":"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"}
Jul 27 22:39:32.771465 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:32.771090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" event={"ID":"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7","Type":"ContainerStarted","Data":"157251edd5a3c27a1136b7120b1eba5addf8669785be7467930ac9d4ffec937f"}
Jul 27 22:39:32.791288 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:32.791234    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" podStartSLOduration=1.7912171799999999 podStartE2EDuration="1.79121718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:39:32.78933706 +0000 UTC m=+2682.976177720" watchObservedRunningTime="2026-07-27 22:39:32.79121718 +0000 UTC m=+2682.978057842"
Jul 27 22:39:34.547319 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:34.546610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:39:35.415150 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.415086    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" podUID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" containerName="main" containerID="cri-o://4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0" gracePeriod=45
Jul 27 22:39:35.671938 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.671886    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:39:35.790556 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.790520    2578 generic.go:356] "Generic (PLEG): container finished" podID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" containerID="4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0" exitCode=0
Jul 27 22:39:35.790761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.790578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"
Jul 27 22:39:35.790761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.790613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" event={"ID":"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3","Type":"ContainerDied","Data":"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"}
Jul 27 22:39:35.790761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.790661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d" event={"ID":"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3","Type":"ContainerDied","Data":"b38ac03569e4acb4b1c9feb661d3f419c8334e3c48c23db65585efd56fdb5aed"}
Jul 27 22:39:35.790761 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.790706    2578 scope.go:117] "RemoveContainer" containerID="4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"
Jul 27 22:39:35.799577 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.799560    2578 scope.go:117] "RemoveContainer" containerID="4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"
Jul 27 22:39:35.799867 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:39:35.799847    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0\": container with ID starting with 4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0 not found: ID does not exist" containerID="4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"
Jul 27 22:39:35.799968 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.799875    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0"} err="failed to get container status \"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0\": rpc error: code = NotFound desc = could not find container \"4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0\": container with ID starting with 4ee0113a13b230504952abac39a2a8af2df0268a3bcf8112dac2df487ec1f0a0 not found: ID does not exist"
Jul 27 22:39:35.841325 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q688\" (UniqueName: \"kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841430 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841352    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841473 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841435    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841473 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841546 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841486    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841546 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841504    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home\") pod \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\" (UID: \"2b5ef4b7-074c-45a7-ac4f-a8cec48081f3\") "
Jul 27 22:39:35.841718 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841693    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:39:35.841814 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841791    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:39:35.841909 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841880    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home" (OuterVolumeSpecName: "home") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:39:35.841969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841902    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:35.841969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.841930    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:35.843635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.843613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm" (OuterVolumeSpecName: "dshm") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:39:35.843635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.843619    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:39:35.843779 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.843712    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688" (OuterVolumeSpecName: "kube-api-access-5q688") pod "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" (UID: "2b5ef4b7-074c-45a7-ac4f-a8cec48081f3"). InnerVolumeSpecName "kube-api-access-5q688". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:39:35.943401 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.943317    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q688\" (UniqueName: \"kubernetes.io/projected/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-kube-api-access-5q688\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:35.943401 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.943350    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:35.943401 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.943362    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:35.943401 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:35.943369    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:39:36.114526 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:36.114495    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:39:36.119198 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:36.119174    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/auth-test-model-c5cbe3-kserve-6755cf854c-rnh6d"]
Jul 27 22:39:36.323266 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:36.323233    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" path="/var/lib/kubelet/pods/2b5ef4b7-074c-45a7-ac4f-a8cec48081f3/volumes"
Jul 27 22:39:41.960317 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:41.960203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:41.960317 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:41.960275    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:41.972625 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:41.972599    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:42.838919 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:42.838893    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:39:50.386459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:50.386433    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:39:50.397913 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:50.397889    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:39:50.684438 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:39:50.684352    2578 scope.go:117] "RemoveContainer" containerID="e308efafbb7f901c150101fe4af682197d26125e011e567c8ad18d12f8bc3241"
Jul 27 22:40:05.748745 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:40:05.748700    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:40:39.845689 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:40:39.845630    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:40:56.937480 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:40:56.937450    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:41:09.918246 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:09.918214    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:41:11.855526 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:11.855448    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:41:15.833185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.833142    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:41:15.833776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.833755    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" containerName="main"
Jul 27 22:41:15.833776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.833777    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" containerName="main"
Jul 27 22:41:15.833924 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.833895    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b5ef4b7-074c-45a7-ac4f-a8cec48081f3" containerName="main"
Jul 27 22:41:15.837395 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.837375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.840099 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.840074    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c5cbe3\"/\"rate-test-model-c5cbe3-kserve-self-signed-certs\""
Jul 27 22:41:15.844830 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.844810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:41:15.949340 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949306    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.949497 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.949497 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnk4n\" (UniqueName: \"kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.949497 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.949497 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:15.949638 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:15.949512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.050924 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.050897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051078 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.050970    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051078 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.050997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnk4n\" (UniqueName: \"kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051184 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051184 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051184 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051479 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.051605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.051551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.053461 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.053444    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.053565 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.053537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.060866 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.060842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnk4n\" (UniqueName: \"kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n\") pod \"rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") " pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.147414 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.147337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:16.289421 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.289392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:41:16.291734 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:41:16.291704    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e43e665_779d_432c_aa58_a2f2a3601edf.slice/crio-9b7d21343beb593388b395a6ab91f254458f3dce2380000da1a0ef6796eee4b9 WatchSource:0}: Error finding container 9b7d21343beb593388b395a6ab91f254458f3dce2380000da1a0ef6796eee4b9: Status 404 returned error can't find the container with id 9b7d21343beb593388b395a6ab91f254458f3dce2380000da1a0ef6796eee4b9
Jul 27 22:41:16.315852 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:16.315825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" event={"ID":"7e43e665-779d-432c-aa58-a2f2a3601edf","Type":"ContainerStarted","Data":"9b7d21343beb593388b395a6ab91f254458f3dce2380000da1a0ef6796eee4b9"}
Jul 27 22:41:17.322319 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:17.322283    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" event={"ID":"7e43e665-779d-432c-aa58-a2f2a3601edf","Type":"ContainerStarted","Data":"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"}
Jul 27 22:41:17.343103 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:17.343052    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" podStartSLOduration=2.343038112 podStartE2EDuration="2.343038112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:41:17.341581522 +0000 UTC m=+2787.528422181" watchObservedRunningTime="2026-07-27 22:41:17.343038112 +0000 UTC m=+2787.529878770"
Jul 27 22:41:24.937388 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:24.937327    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" podUID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" containerName="main" containerID="cri-o://8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c" gracePeriod=45
Jul 27 22:41:25.197950 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.197889    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:41:25.336047 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336017    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336075    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336100    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgf26\" (UniqueName: \"kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336144    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336452 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs\") pod \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\" (UID: \"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7\") "
Jul 27 22:41:25.336452 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:41:25.336532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home" (OuterVolumeSpecName: "home") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:41:25.336532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336420    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache" (OuterVolumeSpecName: "model-cache") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:41:25.336716 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336697    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:25.336804 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336722    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:25.336804 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.336737    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:25.338459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.338418    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm" (OuterVolumeSpecName: "dshm") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:41:25.338459 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.338428    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26" (OuterVolumeSpecName: "kube-api-access-lgf26") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "kube-api-access-lgf26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:41:25.338601 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.338488    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" (UID: "df2ab7ea-9ee3-4235-88dd-8bc32c406bd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:41:25.362646 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.362615    2578 generic.go:356] "Generic (PLEG): container finished" podID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" containerID="8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c" exitCode=0
Jul 27 22:41:25.362780 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.362656    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"
Jul 27 22:41:25.362780 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.362665    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" event={"ID":"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7","Type":"ContainerDied","Data":"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"}
Jul 27 22:41:25.362892 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.362797    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9" event={"ID":"df2ab7ea-9ee3-4235-88dd-8bc32c406bd7","Type":"ContainerDied","Data":"157251edd5a3c27a1136b7120b1eba5addf8669785be7467930ac9d4ffec937f"}
Jul 27 22:41:25.362892 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.362816    2578 scope.go:117] "RemoveContainer" containerID="8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"
Jul 27 22:41:25.374158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.374133    2578 scope.go:117] "RemoveContainer" containerID="8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"
Jul 27 22:41:25.374413 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:41:25.374396    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c\": container with ID starting with 8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c not found: ID does not exist" containerID="8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"
Jul 27 22:41:25.374465 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.374421    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c"} err="failed to get container status \"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c\": rpc error: code = NotFound desc = could not find container \"8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c\": container with ID starting with 8164209c6151ca54a7b1df8ee891b03e5af23d940e80730ca6fe4e27615ab00c not found: ID does not exist"
Jul 27 22:41:25.389215 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.389183    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:41:25.393962 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.393939    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/sub-test-model-c5cbe3-kserve-7b67995f7-6m6d9"]
Jul 27 22:41:25.437869 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.437844    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:25.437869 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.437866    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgf26\" (UniqueName: \"kubernetes.io/projected/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-kube-api-access-lgf26\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:25.438005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:25.437876    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:41:26.148054 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:26.148026    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:26.148455 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:26.148068    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:26.160611 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:26.160587    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:26.324357 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:26.324324    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" path="/var/lib/kubelet/pods/df2ab7ea-9ee3-4235-88dd-8bc32c406bd7/volumes"
Jul 27 22:41:26.380282 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:26.380251    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:41:27.248917 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:27.248887    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:41:39.046942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:41:39.046910    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:42:09.049452 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:09.049415    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:42:27.253104 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:27.253070    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:42:46.843228 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:46.843155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:42:53.590643 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:53.590610    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:42:58.276050 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.276017    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7588-140677006777136"]
Jul 27 22:42:58.276508 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.276451    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" containerName="main"
Jul 27 22:42:58.276508 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.276464    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" containerName="main"
Jul 27 22:42:58.276582 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.276541    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2ab7ea-9ee3-4235-88dd-8bc32c406bd7" containerName="main"
Jul 27 22:42:58.279640 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.279625    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:42:58.282226 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.282209    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gh969\""
Jul 27 22:42:58.286808 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.286787    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7588-140677006777136"]
Jul 27 22:42:58.388105 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.388070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvcc\" (UniqueName: \"kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc\") pod \"test-curl-7588-140677006777136\" (UID: \"950b6819-c28f-4573-b441-58fcc1516408\") " pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:42:58.489366 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.489332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvcc\" (UniqueName: \"kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc\") pod \"test-curl-7588-140677006777136\" (UID: \"950b6819-c28f-4573-b441-58fcc1516408\") " pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:42:58.498574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.498541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvcc\" (UniqueName: \"kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc\") pod \"test-curl-7588-140677006777136\" (UID: \"950b6819-c28f-4573-b441-58fcc1516408\") " pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:42:58.590453 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.590377    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:42:58.724043 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.724019    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7588-140677006777136"]
Jul 27 22:42:58.726786 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:42:58.726755    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod950b6819_c28f_4573_b441_58fcc1516408.slice/crio-ab354daf2a283b0b54e39980dfe6c47258857781aabd66378a9703b8d14f0214 WatchSource:0}: Error finding container ab354daf2a283b0b54e39980dfe6c47258857781aabd66378a9703b8d14f0214: Status 404 returned error can't find the container with id ab354daf2a283b0b54e39980dfe6c47258857781aabd66378a9703b8d14f0214
Jul 27 22:42:58.728111 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.728092    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 22:42:58.837925 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:58.837894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7588-140677006777136" event={"ID":"950b6819-c28f-4573-b441-58fcc1516408","Type":"ContainerStarted","Data":"ab354daf2a283b0b54e39980dfe6c47258857781aabd66378a9703b8d14f0214"}
Jul 27 22:42:59.844973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:59.844815    2578 generic.go:356] "Generic (PLEG): container finished" podID="950b6819-c28f-4573-b441-58fcc1516408" containerID="f9b071cddbed914139b69b2629621c0826ef5a16f72fa6092cc3a13e9a9d4f66" exitCode=0
Jul 27 22:42:59.844973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:59.844894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7588-140677006777136" event={"ID":"950b6819-c28f-4573-b441-58fcc1516408","Type":"ContainerDied","Data":"f9b071cddbed914139b69b2629621c0826ef5a16f72fa6092cc3a13e9a9d4f66"}
Jul 27 22:42:59.847303 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:42:59.847280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:43:00.990996 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:00.990967    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:43:01.031916 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.031891    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7588-140677006777136_950b6819-c28f-4573-b441-58fcc1516408/test-curl-7588-140677006777136/0.log"
Jul 27 22:43:01.054354 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.054329    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7588-140677006777136"]
Jul 27 22:43:01.060535 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.060512    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7588-140677006777136"]
Jul 27 22:43:01.112298 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.112219    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nvcc\" (UniqueName: \"kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc\") pod \"950b6819-c28f-4573-b441-58fcc1516408\" (UID: \"950b6819-c28f-4573-b441-58fcc1516408\") "
Jul 27 22:43:01.114606 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.114583    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc" (OuterVolumeSpecName: "kube-api-access-7nvcc") pod "950b6819-c28f-4573-b441-58fcc1516408" (UID: "950b6819-c28f-4573-b441-58fcc1516408"). InnerVolumeSpecName "kube-api-access-7nvcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:43:01.210730 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.210697    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7588-140677006985088"]
Jul 27 22:43:01.211117 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.211105    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="950b6819-c28f-4573-b441-58fcc1516408" containerName="test-curl-7588-140677006777136"
Jul 27 22:43:01.211173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.211119    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="950b6819-c28f-4573-b441-58fcc1516408" containerName="test-curl-7588-140677006777136"
Jul 27 22:43:01.211216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.211181    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="950b6819-c28f-4573-b441-58fcc1516408" containerName="test-curl-7588-140677006777136"
Jul 27 22:43:01.213065 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.213044    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nvcc\" (UniqueName: \"kubernetes.io/projected/950b6819-c28f-4573-b441-58fcc1516408-kube-api-access-7nvcc\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:01.214287 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.214272    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:01.222616 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.222594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7588-140677006985088"]
Jul 27 22:43:01.313567 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.313538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf49x\" (UniqueName: \"kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x\") pod \"test-curl-7588-140677006985088\" (UID: \"ee232e12-de15-404d-a88f-ae6c7fef928a\") " pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:01.414815 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.414737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf49x\" (UniqueName: \"kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x\") pod \"test-curl-7588-140677006985088\" (UID: \"ee232e12-de15-404d-a88f-ae6c7fef928a\") " pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:01.424344 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.424319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf49x\" (UniqueName: \"kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x\") pod \"test-curl-7588-140677006985088\" (UID: \"ee232e12-de15-404d-a88f-ae6c7fef928a\") " pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:01.523497 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.523468    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:01.669699 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.669651    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7588-140677006985088"]
Jul 27 22:43:01.671147 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:43:01.671120    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee232e12_de15_404d_a88f_ae6c7fef928a.slice/crio-85a0640da47d5c9b6baf71031f9edd37609f48313ed1542d5080e9d5041fa7d3 WatchSource:0}: Error finding container 85a0640da47d5c9b6baf71031f9edd37609f48313ed1542d5080e9d5041fa7d3: Status 404 returned error can't find the container with id 85a0640da47d5c9b6baf71031f9edd37609f48313ed1542d5080e9d5041fa7d3
Jul 27 22:43:01.856157 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.856122    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:43:01.856157 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.856156    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab354daf2a283b0b54e39980dfe6c47258857781aabd66378a9703b8d14f0214"
Jul 27 22:43:01.857472 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:01.857447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7588-140677006985088" event={"ID":"ee232e12-de15-404d-a88f-ae6c7fef928a","Type":"ContainerStarted","Data":"85a0640da47d5c9b6baf71031f9edd37609f48313ed1542d5080e9d5041fa7d3"}
Jul 27 22:43:01.867399 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:43:01.867373    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7588-140677006777136\" is forbidden: User \"system:node:ip-10-0-136-91.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-91.ec2.internal' and this object" podUID="950b6819-c28f-4573-b441-58fcc1516408" pod="opendatahub/test-curl-7588-140677006777136"
Jul 27 22:43:02.322901 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:02.322867    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="950b6819-c28f-4573-b441-58fcc1516408" path="/var/lib/kubelet/pods/950b6819-c28f-4573-b441-58fcc1516408/volumes"
Jul 27 22:43:02.863687 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:02.863640    2578 generic.go:356] "Generic (PLEG): container finished" podID="ee232e12-de15-404d-a88f-ae6c7fef928a" containerID="cf3895e2129f79254fafaeeb77ce0a6136d0f7e3cf7dc4e20ac250e796e3047e" exitCode=0
Jul 27 22:43:02.863876 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:02.863734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7588-140677006985088" event={"ID":"ee232e12-de15-404d-a88f-ae6c7fef928a","Type":"ContainerDied","Data":"cf3895e2129f79254fafaeeb77ce0a6136d0f7e3cf7dc4e20ac250e796e3047e"}
Jul 27 22:43:04.015519 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.015492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:04.038610 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.038580    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7588-140677006985088_ee232e12-de15-404d-a88f-ae6c7fef928a/test-curl-7588-140677006985088/0.log"
Jul 27 22:43:04.040742 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.040723    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf49x\" (UniqueName: \"kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x\") pod \"ee232e12-de15-404d-a88f-ae6c7fef928a\" (UID: \"ee232e12-de15-404d-a88f-ae6c7fef928a\") "
Jul 27 22:43:04.042908 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.042882    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x" (OuterVolumeSpecName: "kube-api-access-pf49x") pod "ee232e12-de15-404d-a88f-ae6c7fef928a" (UID: "ee232e12-de15-404d-a88f-ae6c7fef928a"). InnerVolumeSpecName "kube-api-access-pf49x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:43:04.060109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.060080    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7588-140677006985088"]
Jul 27 22:43:04.066498 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.066478    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7588-140677006985088"]
Jul 27 22:43:04.142139 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.142052    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf49x\" (UniqueName: \"kubernetes.io/projected/ee232e12-de15-404d-a88f-ae6c7fef928a-kube-api-access-pf49x\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:04.323392 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.323356    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee232e12-de15-404d-a88f-ae6c7fef928a" path="/var/lib/kubelet/pods/ee232e12-de15-404d-a88f-ae6c7fef928a/volumes"
Jul 27 22:43:04.875913 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.875885    2578 scope.go:117] "RemoveContainer" containerID="cf3895e2129f79254fafaeeb77ce0a6136d0f7e3cf7dc4e20ac250e796e3047e"
Jul 27 22:43:04.875913 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:04.875908    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7588-140677006985088"
Jul 27 22:43:08.610208 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.610169    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" podUID="7e43e665-779d-432c-aa58-a2f2a3601edf" containerName="main" containerID="cri-o://abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6" gracePeriod=45
Jul 27 22:43:08.870584 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.870527    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:43:08.889173 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889142    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnk4n\" (UniqueName: \"kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889203    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889236    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889292    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889332    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home\") pod \"7e43e665-779d-432c-aa58-a2f2a3601edf\" (UID: \"7e43e665-779d-432c-aa58-a2f2a3601edf\") "
Jul 27 22:43:08.889956 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889749    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:43:08.889956 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:43:08.889956 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.889920    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home" (OuterVolumeSpecName: "home") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:43:08.892236 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.892208    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n" (OuterVolumeSpecName: "kube-api-access-dnk4n") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "kube-api-access-dnk4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:43:08.892408 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.892383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:43:08.892486 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.892416    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm" (OuterVolumeSpecName: "dshm") pod "7e43e665-779d-432c-aa58-a2f2a3601edf" (UID: "7e43e665-779d-432c-aa58-a2f2a3601edf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:43:08.913915 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.913882    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e43e665-779d-432c-aa58-a2f2a3601edf" containerID="abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6" exitCode=0
Jul 27 22:43:08.914052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.913940    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"
Jul 27 22:43:08.914052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.913960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" event={"ID":"7e43e665-779d-432c-aa58-a2f2a3601edf","Type":"ContainerDied","Data":"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"}
Jul 27 22:43:08.914052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.914015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6" event={"ID":"7e43e665-779d-432c-aa58-a2f2a3601edf","Type":"ContainerDied","Data":"9b7d21343beb593388b395a6ab91f254458f3dce2380000da1a0ef6796eee4b9"}
Jul 27 22:43:08.914052 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.914035    2578 scope.go:117] "RemoveContainer" containerID="abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"
Jul 27 22:43:08.924545 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.924517    2578 scope.go:117] "RemoveContainer" containerID="abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"
Jul 27 22:43:08.924843 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:43:08.924823    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6\": container with ID starting with abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6 not found: ID does not exist" containerID="abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"
Jul 27 22:43:08.924901 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.924856    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6"} err="failed to get container status \"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6\": rpc error: code = NotFound desc = could not find container \"abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6\": container with ID starting with abfa59312905755086719e8bc29c56ede9d0a5109e8102f522cbcbb9c50e42f6 not found: ID does not exist"
Jul 27 22:43:08.936440 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.936407    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:43:08.941374 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.941353    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c5cbe3/rate-test-model-c5cbe3-kserve-7dddf885dc-jbqc6"]
Jul 27 22:43:08.990574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990553    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:08.990574 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990573    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnk4n\" (UniqueName: \"kubernetes.io/projected/7e43e665-779d-432c-aa58-a2f2a3601edf-kube-api-access-dnk4n\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:08.990740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990583    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:08.990740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990612    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:08.990740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990621    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e43e665-779d-432c-aa58-a2f2a3601edf-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:08.990740 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:08.990629    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e43e665-779d-432c-aa58-a2f2a3601edf-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:43:10.323206 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:10.323170    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e43e665-779d-432c-aa58-a2f2a3601edf" path="/var/lib/kubelet/pods/7e43e665-779d-432c-aa58-a2f2a3601edf/volumes"
Jul 27 22:43:18.119809 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.119777    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:43:18.120231 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120172    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e43e665-779d-432c-aa58-a2f2a3601edf" containerName="main"
Jul 27 22:43:18.120231 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120183    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e43e665-779d-432c-aa58-a2f2a3601edf" containerName="main"
Jul 27 22:43:18.120231 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120198    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee232e12-de15-404d-a88f-ae6c7fef928a" containerName="test-curl-7588-140677006985088"
Jul 27 22:43:18.120231 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120205    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee232e12-de15-404d-a88f-ae6c7fef928a" containerName="test-curl-7588-140677006985088"
Jul 27 22:43:18.120655 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120276    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e43e665-779d-432c-aa58-a2f2a3601edf" containerName="main"
Jul 27 22:43:18.120655 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.120288    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee232e12-de15-404d-a88f-ae6c7fef928a" containerName="test-curl-7588-140677006985088"
Jul 27 22:43:18.124638 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.124619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.127600 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.127583    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-48ceef-dockercfg-p5hjs\""
Jul 27 22:43:18.127931 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.127917    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-48ceef\""
Jul 27 22:43:18.137368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.137347    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:43:18.171428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.171397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.171579 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.171441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z99rk\" (UniqueName: \"kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.188950 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.188914    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:43:18.192638 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.192620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.203638 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.203611    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:43:18.272218 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.272185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.272390 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.272225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.272390 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.272290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z99rk\" (UniqueName: \"kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.272390 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.272351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wsk\" (UniqueName: \"kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.272901 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.272883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.282096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.282073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z99rk\" (UniqueName: \"kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk\") pod \"payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.373621 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.373533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.373804 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.373621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wsk\" (UniqueName: \"kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.374304 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.374286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.383618 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.383544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wsk\" (UniqueName: \"kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk\") pod \"payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.435317 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.435282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.503780 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.503743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.591627 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.591596    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:43:18.592487 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:43:18.592458    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3671523_dd33_4244_beeb_72c6d1eab5aa.slice/crio-2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f WatchSource:0}: Error finding container 2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f: Status 404 returned error can't find the container with id 2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f
Jul 27 22:43:18.658534 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.658512    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:43:18.660327 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:43:18.660301    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fb567d_c7fa_4fc7_96b7_a94d9a096d1b.slice/crio-4f579bf3f90ee1108ee46341284dbfc5355f11e9ba451c3a705570f4802a8463 WatchSource:0}: Error finding container 4f579bf3f90ee1108ee46341284dbfc5355f11e9ba451c3a705570f4802a8463: Status 404 returned error can't find the container with id 4f579bf3f90ee1108ee46341284dbfc5355f11e9ba451c3a705570f4802a8463
Jul 27 22:43:18.969433 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.969343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" event={"ID":"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b","Type":"ContainerStarted","Data":"93ff36893a05e960a2cfa504ca3f4e8527d4e299713cc79221a470add359ea3f"}
Jul 27 22:43:18.969433 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.969393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" event={"ID":"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b","Type":"ContainerStarted","Data":"4f579bf3f90ee1108ee46341284dbfc5355f11e9ba451c3a705570f4802a8463"}
Jul 27 22:43:18.969728 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.969449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:18.971096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.971071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" event={"ID":"d3671523-dd33-4244-beeb-72c6d1eab5aa","Type":"ContainerStarted","Data":"2d3bfed714ae943e24457ebdb956665330f8ffe8707cabc17e627b95707140c5"}
Jul 27 22:43:18.971232 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.971112    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:18.971232 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.971128    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" event={"ID":"d3671523-dd33-4244-beeb-72c6d1eab5aa","Type":"ContainerStarted","Data":"2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f"}
Jul 27 22:43:18.991752 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:18.991700    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" podStartSLOduration=0.991650082 podStartE2EDuration="991.650082ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:43:18.991308679 +0000 UTC m=+2909.178149338" watchObservedRunningTime="2026-07-27 22:43:18.991650082 +0000 UTC m=+2909.178490744"
Jul 27 22:43:19.009351 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:19.009299    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" podStartSLOduration=1.009282986 podStartE2EDuration="1.009282986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:43:19.008129636 +0000 UTC m=+2909.194970296" watchObservedRunningTime="2026-07-27 22:43:19.009282986 +0000 UTC m=+2909.196123646"
Jul 27 22:43:26.341873 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:26.341840    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:43:29.978595 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:29.978564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:43:29.979121 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:29.978635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:43:36.990428 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:36.990390    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:43:36.995696 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:36.995643    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:36.999026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:36.998996    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e47da1\""
Jul 27 22:43:36.999147 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:36.999040    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e47da1-dockercfg-dqjt4\""
Jul 27 22:43:37.006936 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.006911    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:43:37.054256 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.054219    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:43:37.057973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.057955    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.072244 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.072221    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:43:37.160881 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.160848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znwtn\" (UniqueName: \"kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.161053 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.161024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.161116 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.161067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.161171 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.161130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9tvs\" (UniqueName: \"kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.262539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.262445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.262539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.262481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.262539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.262507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9tvs\" (UniqueName: \"kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.262539 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.262532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znwtn\" (UniqueName: \"kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.263311 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.263287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.263397 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.263287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.274698 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.274654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9tvs\" (UniqueName: \"kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs\") pod \"payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.274797 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.274654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znwtn\" (UniqueName: \"kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn\") pod \"payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.307645 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.307604    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:37.368394 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.368355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:37.463431 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:43:37.463396    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28494ae7_6839_4b91_a950_921492aaa344.slice/crio-ef47d3feb7fb359c7f70e599554807a19cdb6dd5f306cebb50688c9a95c5f962 WatchSource:0}: Error finding container ef47d3feb7fb359c7f70e599554807a19cdb6dd5f306cebb50688c9a95c5f962: Status 404 returned error can't find the container with id ef47d3feb7fb359c7f70e599554807a19cdb6dd5f306cebb50688c9a95c5f962
Jul 27 22:43:37.464175 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.464132    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:43:37.740856 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:37.740834    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:43:37.743431 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:43:37.743402    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fac5aea_7c9c_4062_bfd9_d3c903c22c87.slice/crio-af555b00d53cfa418e164f32d5ad4f49cfce3e017142045ad9d346f43c9b7d9e WatchSource:0}: Error finding container af555b00d53cfa418e164f32d5ad4f49cfce3e017142045ad9d346f43c9b7d9e: Status 404 returned error can't find the container with id af555b00d53cfa418e164f32d5ad4f49cfce3e017142045ad9d346f43c9b7d9e
Jul 27 22:43:38.090788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.090621    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" event={"ID":"28494ae7-6839-4b91-a950-921492aaa344","Type":"ContainerStarted","Data":"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"}
Jul 27 22:43:38.091393 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.091328    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:38.091601 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.091562    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" event={"ID":"28494ae7-6839-4b91-a950-921492aaa344","Type":"ContainerStarted","Data":"ef47d3feb7fb359c7f70e599554807a19cdb6dd5f306cebb50688c9a95c5f962"}
Jul 27 22:43:38.092499 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.092475    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" event={"ID":"5fac5aea-7c9c-4062-bfd9-d3c903c22c87","Type":"ContainerStarted","Data":"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"}
Jul 27 22:43:38.092601 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.092524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" event={"ID":"5fac5aea-7c9c-4062-bfd9-d3c903c22c87","Type":"ContainerStarted","Data":"af555b00d53cfa418e164f32d5ad4f49cfce3e017142045ad9d346f43c9b7d9e"}
Jul 27 22:43:38.092601 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.092556    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:38.189929 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.189863    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" podStartSLOduration=2.189838565 podStartE2EDuration="2.189838565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:43:38.152104109 +0000 UTC m=+2928.338944768" watchObservedRunningTime="2026-07-27 22:43:38.189838565 +0000 UTC m=+2928.376679226"
Jul 27 22:43:38.190823 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:38.190781    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" podStartSLOduration=1.190768636 podStartE2EDuration="1.190768636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:43:38.188032681 +0000 UTC m=+2928.374873342" watchObservedRunningTime="2026-07-27 22:43:38.190768636 +0000 UTC m=+2928.377609297"
Jul 27 22:43:46.644044 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:46.644001    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:43:49.100776 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:49.100744    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:43:50.112717 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:50.112685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:43:57.747221 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:43:57.747191    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c_28494ae7-6839-4b91-a950-921492aaa344/payload-processing/0.log"
Jul 27 22:44:03.734765 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.734727    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:44:03.738432 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.738416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.742200 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.742175    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-48ceef\"/\"ipp-route-48ceef-kserve-self-signed-certs\""
Jul 27 22:44:03.742327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.742204    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-48ceef\"/\"kube-root-ca.crt\""
Jul 27 22:44:03.742327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.742233    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-48ceef\"/\"openshift-service-ca.crt\""
Jul 27 22:44:03.742327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.742212    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-48ceef\"/\"default-dockercfg-lsb9j\""
Jul 27 22:44:03.751295 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.751275    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:44:03.800447 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.800596 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrf9\" (UniqueName: \"kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.800596 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.800718 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.800718 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.800785 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.800727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.901989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrf9\" (UniqueName: \"kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902230 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902507 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902507 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.902609 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.902508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.904527 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.904499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.904715 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.904699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:03.916158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:03.916137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrf9\" (UniqueName: \"kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9\") pod \"ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") " pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:04.047322 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:04.047238    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:04.186833 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:04.186733    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:44:04.189775 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:44:04.189741    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95b28638_d74a_4586_872a_5772175f91ec.slice/crio-1aca8fc7be5b6e5a011746c848cd730679aeab45c139d93e474a55329847256a WatchSource:0}: Error finding container 1aca8fc7be5b6e5a011746c848cd730679aeab45c139d93e474a55329847256a: Status 404 returned error can't find the container with id 1aca8fc7be5b6e5a011746c848cd730679aeab45c139d93e474a55329847256a
Jul 27 22:44:04.239747 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:04.239717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" event={"ID":"95b28638-d74a-4586-872a-5772175f91ec","Type":"ContainerStarted","Data":"1aca8fc7be5b6e5a011746c848cd730679aeab45c139d93e474a55329847256a"}
Jul 27 22:44:05.246882 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:05.246848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" event={"ID":"95b28638-d74a-4586-872a-5772175f91ec","Type":"ContainerStarted","Data":"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"}
Jul 27 22:44:05.264989 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:05.264945    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" podStartSLOduration=2.264931256 podStartE2EDuration="2.264931256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:44:05.263914053 +0000 UTC m=+2955.450754713" watchObservedRunningTime="2026-07-27 22:44:05.264931256 +0000 UTC m=+2955.451771912"
Jul 27 22:44:08.050496 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:08.050456    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:44:14.047844 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:14.047756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:14.047844 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:14.047805    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:14.060285 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:14.060260    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:14.308739 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:14.308641    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:44:48.451085 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:48.451054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:44:50.419758 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:50.419728    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:44:50.431945 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:50.431924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:44:59.908001 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:44:59.907974    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk_d3671523-dd33-4244-beeb-72c6d1eab5aa/payload-processing/0.log"
Jul 27 22:45:00.159935 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:00.159856    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c_28494ae7-6839-4b91-a950-921492aaa344/payload-processing/0.log"
Jul 27 22:45:04.496123 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.496087    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:04.499839 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.499816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.502593 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.502570    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ea5c1d\""
Jul 27 22:45:04.502768 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.502622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ea5c1d-dockercfg-vb7q5\""
Jul 27 22:45:04.511243 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.511219    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:04.530188 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.530151    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:04.534498 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.534479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.542562 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.542540    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:04.562992 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.562956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgzm\" (UniqueName: \"kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.563164 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.563064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.664302 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.664267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.664488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.664343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.664488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.664384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgzm\" (UniqueName: \"kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.664488 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.664400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqwm\" (UniqueName: \"kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.665031 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.665010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.673045 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.673026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgzm\" (UniqueName: \"kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm\") pod \"payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.765165 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.765076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.765330 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.765185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqwm\" (UniqueName: \"kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.765840 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.765821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.773401 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.773369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqwm\" (UniqueName: \"kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm\") pod \"payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.811329 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.811297    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:04.847098 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.847066    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:04.983194 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:04.983133    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:04.985862 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:45:04.984860    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87caceef_c258_493a_bab1_1adfc0a4d0bf.slice/crio-8c2b5693a547a2bce059d67bc74b06919eaf990b9945d6fc06e1947a73a8ceb5 WatchSource:0}: Error finding container 8c2b5693a547a2bce059d67bc74b06919eaf990b9945d6fc06e1947a73a8ceb5: Status 404 returned error can't find the container with id 8c2b5693a547a2bce059d67bc74b06919eaf990b9945d6fc06e1947a73a8ceb5
Jul 27 22:45:05.027148 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.027127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:05.029182 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:45:05.029161    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbd670a_09a9_4220_b7af_585983eca63e.slice/crio-251df2c3e7ab38e90dea0cd1fc1d54bc6e9939d2b49ca2d8e5470c78e3bee3b3 WatchSource:0}: Error finding container 251df2c3e7ab38e90dea0cd1fc1d54bc6e9939d2b49ca2d8e5470c78e3bee3b3: Status 404 returned error can't find the container with id 251df2c3e7ab38e90dea0cd1fc1d54bc6e9939d2b49ca2d8e5470c78e3bee3b3
Jul 27 22:45:05.566287 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.566253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" event={"ID":"87caceef-c258-493a-bab1-1adfc0a4d0bf","Type":"ContainerStarted","Data":"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"}
Jul 27 22:45:05.566759 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.566305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" event={"ID":"87caceef-c258-493a-bab1-1adfc0a4d0bf","Type":"ContainerStarted","Data":"8c2b5693a547a2bce059d67bc74b06919eaf990b9945d6fc06e1947a73a8ceb5"}
Jul 27 22:45:05.566759 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.566355    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:05.567969 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.567945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" event={"ID":"fdbd670a-09a9-4220-b7af-585983eca63e","Type":"ContainerStarted","Data":"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"}
Jul 27 22:45:05.568067 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.567986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" event={"ID":"fdbd670a-09a9-4220-b7af-585983eca63e","Type":"ContainerStarted","Data":"251df2c3e7ab38e90dea0cd1fc1d54bc6e9939d2b49ca2d8e5470c78e3bee3b3"}
Jul 27 22:45:05.568067 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.568034    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:05.584005 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.583944    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" podStartSLOduration=1.583922142 podStartE2EDuration="1.583922142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:45:05.582106622 +0000 UTC m=+3015.768947280" watchObservedRunningTime="2026-07-27 22:45:05.583922142 +0000 UTC m=+3015.770762803"
Jul 27 22:45:05.598315 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:05.598253    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" podStartSLOduration=1.598231847 podStartE2EDuration="1.598231847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 22:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:45:05.597916145 +0000 UTC m=+3015.784756805" watchObservedRunningTime="2026-07-27 22:45:05.598231847 +0000 UTC m=+3015.785072509"
Jul 27 22:45:07.852327 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:07.852291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:45:16.575476 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:16.575443    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:16.576069 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:16.575660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:29.597921 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.597887    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:29.598474 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.598165    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" podUID="fdbd670a-09a9-4220-b7af-585983eca63e" containerName="payload-processing" containerID="cri-o://59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403" gracePeriod=30
Jul 27 22:45:29.608385 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.607901    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:29.608385 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.608126    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" podUID="87caceef-c258-493a-bab1-1adfc0a4d0bf" containerName="payload-pre-processing" containerID="cri-o://4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9" gracePeriod=30
Jul 27 22:45:29.876799 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.876774    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:29.892184 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.892161    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:29.902742 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.902720    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqgzm\" (UniqueName: \"kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm\") pod \"87caceef-c258-493a-bab1-1adfc0a4d0bf\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") "
Jul 27 22:45:29.902881 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.902823    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume\") pod \"87caceef-c258-493a-bab1-1adfc0a4d0bf\" (UID: \"87caceef-c258-493a-bab1-1adfc0a4d0bf\") "
Jul 27 22:45:29.903205 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.903184    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87caceef-c258-493a-bab1-1adfc0a4d0bf" (UID: "87caceef-c258-493a-bab1-1adfc0a4d0bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:45:29.905051 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:29.905017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm" (OuterVolumeSpecName: "kube-api-access-jqgzm") pod "87caceef-c258-493a-bab1-1adfc0a4d0bf" (UID: "87caceef-c258-493a-bab1-1adfc0a4d0bf"). InnerVolumeSpecName "kube-api-access-jqgzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:45:30.003733 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.003662    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgqwm\" (UniqueName: \"kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm\") pod \"fdbd670a-09a9-4220-b7af-585983eca63e\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") "
Jul 27 22:45:30.003922 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.003826    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume\") pod \"fdbd670a-09a9-4220-b7af-585983eca63e\" (UID: \"fdbd670a-09a9-4220-b7af-585983eca63e\") "
Jul 27 22:45:30.004026 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.004010    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87caceef-c258-493a-bab1-1adfc0a4d0bf-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:45:30.004158 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.004030    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqgzm\" (UniqueName: \"kubernetes.io/projected/87caceef-c258-493a-bab1-1adfc0a4d0bf-kube-api-access-jqgzm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:45:30.004242 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.004215    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdbd670a-09a9-4220-b7af-585983eca63e" (UID: "fdbd670a-09a9-4220-b7af-585983eca63e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:45:30.005947 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.005922    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm" (OuterVolumeSpecName: "kube-api-access-mgqwm") pod "fdbd670a-09a9-4220-b7af-585983eca63e" (UID: "fdbd670a-09a9-4220-b7af-585983eca63e"). InnerVolumeSpecName "kube-api-access-mgqwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:45:30.105044 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.104967    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbd670a-09a9-4220-b7af-585983eca63e-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:45:30.105044 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.104995    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgqwm\" (UniqueName: \"kubernetes.io/projected/fdbd670a-09a9-4220-b7af-585983eca63e-kube-api-access-mgqwm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:45:30.716862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.716771    2578 generic.go:356] "Generic (PLEG): container finished" podID="fdbd670a-09a9-4220-b7af-585983eca63e" containerID="59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403" exitCode=0
Jul 27 22:45:30.716862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.716825    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"
Jul 27 22:45:30.716862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.716847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" event={"ID":"fdbd670a-09a9-4220-b7af-585983eca63e","Type":"ContainerDied","Data":"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"}
Jul 27 22:45:30.717397 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.716921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59" event={"ID":"fdbd670a-09a9-4220-b7af-585983eca63e","Type":"ContainerDied","Data":"251df2c3e7ab38e90dea0cd1fc1d54bc6e9939d2b49ca2d8e5470c78e3bee3b3"}
Jul 27 22:45:30.717397 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.716943    2578 scope.go:117] "RemoveContainer" containerID="59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"
Jul 27 22:45:30.718543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.718526    2578 generic.go:356] "Generic (PLEG): container finished" podID="87caceef-c258-493a-bab1-1adfc0a4d0bf" containerID="4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9" exitCode=0
Jul 27 22:45:30.718629 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.718558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" event={"ID":"87caceef-c258-493a-bab1-1adfc0a4d0bf","Type":"ContainerDied","Data":"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"}
Jul 27 22:45:30.718629 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.718574    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"
Jul 27 22:45:30.718629 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.718589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth" event={"ID":"87caceef-c258-493a-bab1-1adfc0a4d0bf","Type":"ContainerDied","Data":"8c2b5693a547a2bce059d67bc74b06919eaf990b9945d6fc06e1947a73a8ceb5"}
Jul 27 22:45:30.726651 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.726627    2578 scope.go:117] "RemoveContainer" containerID="59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"
Jul 27 22:45:30.726985 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:45:30.726959    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403\": container with ID starting with 59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403 not found: ID does not exist" containerID="59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"
Jul 27 22:45:30.727072 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.727013    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403"} err="failed to get container status \"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403\": rpc error: code = NotFound desc = could not find container \"59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403\": container with ID starting with 59db8dff7d6391bba1f8aa5db9f0dd43a11d5b903d4affd540b146c08b560403 not found: ID does not exist"
Jul 27 22:45:30.727072 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.727050    2578 scope.go:117] "RemoveContainer" containerID="4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"
Jul 27 22:45:30.735143 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.735124    2578 scope.go:117] "RemoveContainer" containerID="4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"
Jul 27 22:45:30.735414 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:45:30.735395    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9\": container with ID starting with 4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9 not found: ID does not exist" containerID="4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"
Jul 27 22:45:30.735480 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.735423    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9"} err="failed to get container status \"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9\": rpc error: code = NotFound desc = could not find container \"4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9\": container with ID starting with 4b9310ef525a4570c75e050b2e532a1aefc83c3fd5c680f822c6f9a33639a4f9 not found: ID does not exist"
Jul 27 22:45:30.736469 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.736452    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:30.740787 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.740768    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ea5c1d-6c86fc64d4-6cjth"]
Jul 27 22:45:30.752253 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.752230    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:30.757724 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:30.757707    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ea5c1d-67dcf48ff8-2gh59"]
Jul 27 22:45:32.323410 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:32.323379    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87caceef-c258-493a-bab1-1adfc0a4d0bf" path="/var/lib/kubelet/pods/87caceef-c258-493a-bab1-1adfc0a4d0bf/volumes"
Jul 27 22:45:32.323818 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:32.323764    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdbd670a-09a9-4220-b7af-585983eca63e" path="/var/lib/kubelet/pods/fdbd670a-09a9-4220-b7af-585983eca63e/volumes"
Jul 27 22:45:35.741493 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:35.741459    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:45:58.056778 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:58.056738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:45:59.580787 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.580759    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:45:59.581216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.580989    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" podUID="d3671523-dd33-4244-beeb-72c6d1eab5aa" containerName="payload-processing" containerID="cri-o://2d3bfed714ae943e24457ebdb956665330f8ffe8707cabc17e627b95707140c5" gracePeriod=30
Jul 27 22:45:59.589543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.588999    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:45:59.589543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.589213    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" podUID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" containerName="payload-pre-processing" containerID="cri-o://93ff36893a05e960a2cfa504ca3f4e8527d4e299713cc79221a470add359ea3f" gracePeriod=30
Jul 27 22:45:59.874249 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.874222    2578 generic.go:356] "Generic (PLEG): container finished" podID="d3671523-dd33-4244-beeb-72c6d1eab5aa" containerID="2d3bfed714ae943e24457ebdb956665330f8ffe8707cabc17e627b95707140c5" exitCode=0
Jul 27 22:45:59.874369 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.874296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" event={"ID":"d3671523-dd33-4244-beeb-72c6d1eab5aa","Type":"ContainerDied","Data":"2d3bfed714ae943e24457ebdb956665330f8ffe8707cabc17e627b95707140c5"}
Jul 27 22:45:59.874369 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.874361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk" event={"ID":"d3671523-dd33-4244-beeb-72c6d1eab5aa","Type":"ContainerDied","Data":"2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f"}
Jul 27 22:45:59.874473 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.874385    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2293a05b3d77e7b3cfbb198d4cd97777be4ca3a48ef04821e3a3b273cc4e374f"
Jul 27 22:45:59.875816 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.875797    2578 generic.go:356] "Generic (PLEG): container finished" podID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" containerID="93ff36893a05e960a2cfa504ca3f4e8527d4e299713cc79221a470add359ea3f" exitCode=0
Jul 27 22:45:59.875915 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.875872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" event={"ID":"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b","Type":"ContainerDied","Data":"93ff36893a05e960a2cfa504ca3f4e8527d4e299713cc79221a470add359ea3f"}
Jul 27 22:45:59.889442 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.889423    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:45:59.892713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.892692    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:45:59.991486 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991459    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume\") pod \"d3671523-dd33-4244-beeb-72c6d1eab5aa\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") "
Jul 27 22:45:59.991486 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991491    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume\") pod \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") "
Jul 27 22:45:59.991792 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991565    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z99rk\" (UniqueName: \"kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk\") pod \"d3671523-dd33-4244-beeb-72c6d1eab5aa\" (UID: \"d3671523-dd33-4244-beeb-72c6d1eab5aa\") "
Jul 27 22:45:59.991792 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991598    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79wsk\" (UniqueName: \"kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk\") pod \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\" (UID: \"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b\") "
Jul 27 22:45:59.991926 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3671523-dd33-4244-beeb-72c6d1eab5aa" (UID: "d3671523-dd33-4244-beeb-72c6d1eab5aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:45:59.991979 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.991939    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" (UID: "26fb567d-c7fa-4fc7-96b7-a94d9a096d1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:45:59.993829 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.993805    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk" (OuterVolumeSpecName: "kube-api-access-z99rk") pod "d3671523-dd33-4244-beeb-72c6d1eab5aa" (UID: "d3671523-dd33-4244-beeb-72c6d1eab5aa"). InnerVolumeSpecName "kube-api-access-z99rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:45:59.993957 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:45:59.993936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk" (OuterVolumeSpecName: "kube-api-access-79wsk") pod "26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" (UID: "26fb567d-c7fa-4fc7-96b7-a94d9a096d1b"). InnerVolumeSpecName "kube-api-access-79wsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:46:00.092633 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.092600    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z99rk\" (UniqueName: \"kubernetes.io/projected/d3671523-dd33-4244-beeb-72c6d1eab5aa-kube-api-access-z99rk\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:00.092633 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.092627    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79wsk\" (UniqueName: \"kubernetes.io/projected/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-kube-api-access-79wsk\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:00.092633 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.092636    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3671523-dd33-4244-beeb-72c6d1eab5aa-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:00.092894 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.092646    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:00.881754 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.881657    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"
Jul 27 22:46:00.881754 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.881695    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h" event={"ID":"26fb567d-c7fa-4fc7-96b7-a94d9a096d1b","Type":"ContainerDied","Data":"4f579bf3f90ee1108ee46341284dbfc5355f11e9ba451c3a705570f4802a8463"}
Jul 27 22:46:00.882199 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.881763    2578 scope.go:117] "RemoveContainer" containerID="93ff36893a05e960a2cfa504ca3f4e8527d4e299713cc79221a470add359ea3f"
Jul 27 22:46:00.882199 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.881940    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"
Jul 27 22:46:00.902878 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.902850    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:46:00.909073 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.909053    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-48ceef-68446c98d4-sxc4h"]
Jul 27 22:46:00.919505 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.919486    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:46:00.924463 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:00.924444    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48ceef-58d87d5b56-nrbvk"]
Jul 27 22:46:02.324632 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:02.324596    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" path="/var/lib/kubelet/pods/26fb567d-c7fa-4fc7-96b7-a94d9a096d1b/volumes"
Jul 27 22:46:02.325035 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:02.324974    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3671523-dd33-4244-beeb-72c6d1eab5aa" path="/var/lib/kubelet/pods/d3671523-dd33-4244-beeb-72c6d1eab5aa/volumes"
Jul 27 22:46:09.468093 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:09.468061    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:46:09.950391 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:09.950350    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:46:24.490528 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.490483    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" podUID="95b28638-d74a-4586-872a-5772175f91ec" containerName="main" containerID="cri-o://9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9" gracePeriod=45
Jul 27 22:46:24.751491 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.751428    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:46:24.822201 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xrf9\" (UniqueName: \"kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822223    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822268    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822317    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822355 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache\") pod \"95b28638-d74a-4586-872a-5772175f91ec\" (UID: \"95b28638-d74a-4586-872a-5772175f91ec\") "
Jul 27 22:46:24.822642 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822591    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home" (OuterVolumeSpecName: "home") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:46:24.822642 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822601    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:46:24.822758 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.822654    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache" (OuterVolumeSpecName: "model-cache") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:46:24.824585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.824556    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm" (OuterVolumeSpecName: "dshm") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 22:46:24.824715 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.824599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 22:46:24.824715 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.824638    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9" (OuterVolumeSpecName: "kube-api-access-6xrf9") pod "95b28638-d74a-4586-872a-5772175f91ec" (UID: "95b28638-d74a-4586-872a-5772175f91ec"). InnerVolumeSpecName "kube-api-access-6xrf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:46:24.923536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923484    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-home\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:24.923536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923530    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-dshm\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:24.923802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923568    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-tmp-dir\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:24.923802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923580    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95b28638-d74a-4586-872a-5772175f91ec-model-cache\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:24.923802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923593    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xrf9\" (UniqueName: \"kubernetes.io/projected/95b28638-d74a-4586-872a-5772175f91ec-kube-api-access-6xrf9\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:24.923802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:24.923606    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95b28638-d74a-4586-872a-5772175f91ec-tls-certs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:25.020561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.020474    2578 generic.go:356] "Generic (PLEG): container finished" podID="95b28638-d74a-4586-872a-5772175f91ec" containerID="9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9" exitCode=0
Jul 27 22:46:25.020561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.020518    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"
Jul 27 22:46:25.020788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.020559    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" event={"ID":"95b28638-d74a-4586-872a-5772175f91ec","Type":"ContainerDied","Data":"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"}
Jul 27 22:46:25.020788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.020604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq" event={"ID":"95b28638-d74a-4586-872a-5772175f91ec","Type":"ContainerDied","Data":"1aca8fc7be5b6e5a011746c848cd730679aeab45c139d93e474a55329847256a"}
Jul 27 22:46:25.020788 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.020622    2578 scope.go:117] "RemoveContainer" containerID="9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"
Jul 27 22:46:25.030255 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.030208    2578 scope.go:117] "RemoveContainer" containerID="9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"
Jul 27 22:46:25.030503 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:46:25.030484    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9\": container with ID starting with 9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9 not found: ID does not exist" containerID="9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"
Jul 27 22:46:25.030553 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.030512    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9"} err="failed to get container status \"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9\": rpc error: code = NotFound desc = could not find container \"9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9\": container with ID starting with 9b73bff5b96d306807f0bf72522ff48d5be6b3ad7da6bcc5cc43492943d04cc9 not found: ID does not exist"
Jul 27 22:46:25.043272 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.043246    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:46:25.047252 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:25.047225    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-48ceef/ipp-route-48ceef-kserve-5bdbf7f9b7-s85nq"]
Jul 27 22:46:26.323503 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:26.323468    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b28638-d74a-4586-872a-5772175f91ec" path="/var/lib/kubelet/pods/95b28638-d74a-4586-872a-5772175f91ec/volumes"
Jul 27 22:46:52.777894 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.777855    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"]
Jul 27 22:46:52.778461 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778440    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87caceef-c258-493a-bab1-1adfc0a4d0bf" containerName="payload-pre-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778463    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="87caceef-c258-493a-bab1-1adfc0a4d0bf" containerName="payload-pre-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778473    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdbd670a-09a9-4220-b7af-585983eca63e" containerName="payload-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778481    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdbd670a-09a9-4220-b7af-585983eca63e" containerName="payload-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778499    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3671523-dd33-4244-beeb-72c6d1eab5aa" containerName="payload-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778508    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3671523-dd33-4244-beeb-72c6d1eab5aa" containerName="payload-processing"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778521    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b28638-d74a-4586-872a-5772175f91ec" containerName="main"
Jul 27 22:46:52.778547 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778528    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b28638-d74a-4586-872a-5772175f91ec" containerName="main"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778553    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" containerName="payload-pre-processing"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778561    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" containerName="payload-pre-processing"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778663    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdbd670a-09a9-4220-b7af-585983eca63e" containerName="payload-processing"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778696    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26fb567d-c7fa-4fc7-96b7-a94d9a096d1b" containerName="payload-pre-processing"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778712    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b28638-d74a-4586-872a-5772175f91ec" containerName="main"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778722    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3671523-dd33-4244-beeb-72c6d1eab5aa" containerName="payload-processing"
Jul 27 22:46:52.778942 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.778732    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="87caceef-c258-493a-bab1-1adfc0a4d0bf" containerName="payload-pre-processing"
Jul 27 22:46:52.783410 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.783391    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:52.786940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.786767    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:46:52.786940 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.786792    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:46:52.789210 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.789186    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"]
Jul 27 22:46:52.877705 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.875561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp\" (UID: \"b60ac5e4-dc11-48da-ab39-2e74bb3c576c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:52.976748 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.976712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp\" (UID: \"b60ac5e4-dc11-48da-ab39-2e74bb3c576c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:52.977297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:52.977277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp\" (UID: \"b60ac5e4-dc11-48da-ab39-2e74bb3c576c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:53.096492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:53.096404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:53.446081 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:53.446056    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"]
Jul 27 22:46:53.447777 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:46:53.447752    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60ac5e4_dc11_48da_ab39_2e74bb3c576c.slice/crio-b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3 WatchSource:0}: Error finding container b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3: Status 404 returned error can't find the container with id b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3
Jul 27 22:46:54.177078 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:54.177042    2578 generic.go:356] "Generic (PLEG): container finished" podID="b60ac5e4-dc11-48da-ab39-2e74bb3c576c" containerID="6799baeda30c97a07efcd4115411a359657bf2413259f3cd0e3037eef543f6d9" exitCode=0
Jul 27 22:46:54.177505 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:54.177093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp" event={"ID":"b60ac5e4-dc11-48da-ab39-2e74bb3c576c","Type":"ContainerDied","Data":"6799baeda30c97a07efcd4115411a359657bf2413259f3cd0e3037eef543f6d9"}
Jul 27 22:46:54.177505 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:54.177153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp" event={"ID":"b60ac5e4-dc11-48da-ab39-2e74bb3c576c","Type":"ContainerStarted","Data":"b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3"}
Jul 27 22:46:55.330581 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:55.330558    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:55.402915 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:55.402874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca\") pod \"b60ac5e4-dc11-48da-ab39-2e74bb3c576c\" (UID: \"b60ac5e4-dc11-48da-ab39-2e74bb3c576c\") "
Jul 27 22:46:55.403208 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:55.403185    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b60ac5e4-dc11-48da-ab39-2e74bb3c576c" (UID: "b60ac5e4-dc11-48da-ab39-2e74bb3c576c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:46:55.403342 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:55.403326    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60ac5e4-dc11-48da-ab39-2e74bb3c576c-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:46:56.188396 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:56.188361    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"
Jul 27 22:46:56.188646 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:56.188395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp" event={"ID":"b60ac5e4-dc11-48da-ab39-2e74bb3c576c","Type":"ContainerDied","Data":"b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3"}
Jul 27 22:46:56.188646 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:46:56.188455    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8ef83843de1d7eed138ae0f6a524679dc148b6b4a0199722ae9332052666aa3"
Jul 27 22:47:03.279842 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.279800    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:47:03.280318 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.280071    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" podUID="28494ae7-6839-4b91-a950-921492aaa344" containerName="payload-processing" containerID="cri-o://38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003" gracePeriod=30
Jul 27 22:47:03.293149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.293122    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:47:03.293398 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.293375    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" podUID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" containerName="payload-pre-processing" containerID="cri-o://26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98" gracePeriod=30
Jul 27 22:47:03.579139 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.579118    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:47:03.582037 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.582019    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:47:03.676358 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676325    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume\") pod \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") "
Jul 27 22:47:03.676561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676399    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9tvs\" (UniqueName: \"kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs\") pod \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\" (UID: \"5fac5aea-7c9c-4062-bfd9-d3c903c22c87\") "
Jul 27 22:47:03.676561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676491    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume\") pod \"28494ae7-6839-4b91-a950-921492aaa344\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") "
Jul 27 22:47:03.676561 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676535    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znwtn\" (UniqueName: \"kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn\") pod \"28494ae7-6839-4b91-a950-921492aaa344\" (UID: \"28494ae7-6839-4b91-a950-921492aaa344\") "
Jul 27 22:47:03.676841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5fac5aea-7c9c-4062-bfd9-d3c903c22c87" (UID: "5fac5aea-7c9c-4062-bfd9-d3c903c22c87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:03.676955 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.676866    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28494ae7-6839-4b91-a950-921492aaa344" (UID: "28494ae7-6839-4b91-a950-921492aaa344"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:03.678621 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.678598    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs" (OuterVolumeSpecName: "kube-api-access-z9tvs") pod "5fac5aea-7c9c-4062-bfd9-d3c903c22c87" (UID: "5fac5aea-7c9c-4062-bfd9-d3c903c22c87"). InnerVolumeSpecName "kube-api-access-z9tvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:03.678735 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.678653    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn" (OuterVolumeSpecName: "kube-api-access-znwtn") pod "28494ae7-6839-4b91-a950-921492aaa344" (UID: "28494ae7-6839-4b91-a950-921492aaa344"). InnerVolumeSpecName "kube-api-access-znwtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:03.777205 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.777170    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:03.777205 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.777199    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9tvs\" (UniqueName: \"kubernetes.io/projected/5fac5aea-7c9c-4062-bfd9-d3c903c22c87-kube-api-access-z9tvs\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:03.777205 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.777209    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28494ae7-6839-4b91-a950-921492aaa344-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:03.777439 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:03.777218    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znwtn\" (UniqueName: \"kubernetes.io/projected/28494ae7-6839-4b91-a950-921492aaa344-kube-api-access-znwtn\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:04.232396 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.232362    2578 generic.go:356] "Generic (PLEG): container finished" podID="28494ae7-6839-4b91-a950-921492aaa344" containerID="38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003" exitCode=0
Jul 27 22:47:04.232585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.232409    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"
Jul 27 22:47:04.232585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.232434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" event={"ID":"28494ae7-6839-4b91-a950-921492aaa344","Type":"ContainerDied","Data":"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"}
Jul 27 22:47:04.232585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.232490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c" event={"ID":"28494ae7-6839-4b91-a950-921492aaa344","Type":"ContainerDied","Data":"ef47d3feb7fb359c7f70e599554807a19cdb6dd5f306cebb50688c9a95c5f962"}
Jul 27 22:47:04.232585 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.232522    2578 scope.go:117] "RemoveContainer" containerID="38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"
Jul 27 22:47:04.233926 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.233901    2578 generic.go:356] "Generic (PLEG): container finished" podID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" containerID="26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98" exitCode=0
Jul 27 22:47:04.234068 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.234008    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"
Jul 27 22:47:04.234068 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.234028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" event={"ID":"5fac5aea-7c9c-4062-bfd9-d3c903c22c87","Type":"ContainerDied","Data":"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"}
Jul 27 22:47:04.234161 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.234071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2" event={"ID":"5fac5aea-7c9c-4062-bfd9-d3c903c22c87","Type":"ContainerDied","Data":"af555b00d53cfa418e164f32d5ad4f49cfce3e017142045ad9d346f43c9b7d9e"}
Jul 27 22:47:04.241973 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.241958    2578 scope.go:117] "RemoveContainer" containerID="38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"
Jul 27 22:47:04.242220 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:04.242203    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003\": container with ID starting with 38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003 not found: ID does not exist" containerID="38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"
Jul 27 22:47:04.242298 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.242234    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003"} err="failed to get container status \"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003\": rpc error: code = NotFound desc = could not find container \"38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003\": container with ID starting with 38d4db008870f60d0e554a96c5c4f1adafc80c3a0f3bab0bf3d41acf1b301003 not found: ID does not exist"
Jul 27 22:47:04.242298 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.242256    2578 scope.go:117] "RemoveContainer" containerID="26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"
Jul 27 22:47:04.250601 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.250584    2578 scope.go:117] "RemoveContainer" containerID="26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"
Jul 27 22:47:04.250877 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:04.250859    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98\": container with ID starting with 26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98 not found: ID does not exist" containerID="26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"
Jul 27 22:47:04.250944 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.250881    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98"} err="failed to get container status \"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98\": rpc error: code = NotFound desc = could not find container \"26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98\": container with ID starting with 26e083e0e20cbe7f7a010737c3dfa1efb4d584e713bc142cb8de16520e4d2a98 not found: ID does not exist"
Jul 27 22:47:04.256713 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.256656    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:47:04.260425 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.260407    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e47da1-5c45d55f96-cmx9c"]
Jul 27 22:47:04.270851 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.270833    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:47:04.274532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.274512    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e47da1-5c8475dfb4-zc7k2"]
Jul 27 22:47:04.323939 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.323918    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28494ae7-6839-4b91-a950-921492aaa344" path="/var/lib/kubelet/pods/28494ae7-6839-4b91-a950-921492aaa344/volumes"
Jul 27 22:47:04.324262 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:04.324249    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" path="/var/lib/kubelet/pods/5fac5aea-7c9c-4062-bfd9-d3c903c22c87/volumes"
Jul 27 22:47:06.441609 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:06.441571    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:47:07.851235 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:07.851188    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"]
Jul 27 22:47:07.852958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:07.852928    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-e47da1-28qxp"]
Jul 27 22:47:08.322765 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:08.322731    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b60ac5e4-dc11-48da-ab39-2e74bb3c576c" path="/var/lib/kubelet/pods/b60ac5e4-dc11-48da-ab39-2e74bb3c576c/volumes"
Jul 27 22:47:14.636014 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.635936    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"]
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636321    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60ac5e4-dc11-48da-ab39-2e74bb3c576c" containerName="revoke-keys"
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636332    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60ac5e4-dc11-48da-ab39-2e74bb3c576c" containerName="revoke-keys"
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636360    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" containerName="payload-pre-processing"
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636366    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" containerName="payload-pre-processing"
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636373    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28494ae7-6839-4b91-a950-921492aaa344" containerName="payload-processing"
Jul 27 22:47:14.636412 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636379    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="28494ae7-6839-4b91-a950-921492aaa344" containerName="payload-processing"
Jul 27 22:47:14.636635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636435    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fac5aea-7c9c-4062-bfd9-d3c903c22c87" containerName="payload-pre-processing"
Jul 27 22:47:14.636635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636447    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="28494ae7-6839-4b91-a950-921492aaa344" containerName="payload-processing"
Jul 27 22:47:14.636635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.636454    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b60ac5e4-dc11-48da-ab39-2e74bb3c576c" containerName="revoke-keys"
Jul 27 22:47:14.641011 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.640989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:14.643947 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.643924    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:47:14.644960 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.644915    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:47:14.646370 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.646351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"]
Jul 27 22:47:14.790049 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.790012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2\" (UID: \"5141577e-9793-4f22-9a6c-9d89c124e2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:14.891275 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.891179    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2\" (UID: \"5141577e-9793-4f22-9a6c-9d89c124e2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:14.891916 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.891891    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2\" (UID: \"5141577e-9793-4f22-9a6c-9d89c124e2b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:14.953283 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:14.953253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:15.300363 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:15.300328    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"]
Jul 27 22:47:15.301189 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:47:15.301150    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5141577e_9793_4f22_9a6c_9d89c124e2b6.slice/crio-75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7 WatchSource:0}: Error finding container 75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7: Status 404 returned error can't find the container with id 75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7
Jul 27 22:47:16.307088 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:16.306993    2578 generic.go:356] "Generic (PLEG): container finished" podID="5141577e-9793-4f22-9a6c-9d89c124e2b6" containerID="1aac6aff4b49385494e7930a47c83fad8b1557bacd706cdc2de7333ac0dbd96f" exitCode=0
Jul 27 22:47:16.307458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:16.307081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2" event={"ID":"5141577e-9793-4f22-9a6c-9d89c124e2b6","Type":"ContainerDied","Data":"1aac6aff4b49385494e7930a47c83fad8b1557bacd706cdc2de7333ac0dbd96f"}
Jul 27 22:47:16.307458 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:16.307146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2" event={"ID":"5141577e-9793-4f22-9a6c-9d89c124e2b6","Type":"ContainerStarted","Data":"75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7"}
Jul 27 22:47:17.453765 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:17.453738    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:17.617440 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:17.617368    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca\") pod \"5141577e-9793-4f22-9a6c-9d89c124e2b6\" (UID: \"5141577e-9793-4f22-9a6c-9d89c124e2b6\") "
Jul 27 22:47:17.617754 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:17.617732    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5141577e-9793-4f22-9a6c-9d89c124e2b6" (UID: "5141577e-9793-4f22-9a6c-9d89c124e2b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:17.718265 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:17.718235    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141577e-9793-4f22-9a6c-9d89c124e2b6-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:18.318206 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:18.318179    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"
Jul 27 22:47:18.324682 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:18.324643    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2" event={"ID":"5141577e-9793-4f22-9a6c-9d89c124e2b6","Type":"ContainerDied","Data":"75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7"}
Jul 27 22:47:18.324841 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:18.324701    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75ebfbf7f612f11aeaea0ce0610f48ea6795d13d71f423d7a461c852a07283f7"
Jul 27 22:47:25.261875 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.261839    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:47:25.262349 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.262103    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" podUID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" containerName="payload-processing" containerID="cri-o://711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05" gracePeriod=30
Jul 27 22:47:25.268380 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.268021    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:47:25.268380 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.268223    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerName="payload-pre-processing" containerID="cri-o://2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e" gracePeriod=30
Jul 27 22:47:25.551324 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.551300    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:47:25.554149 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.554133    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:47:25.692390 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692357    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume\") pod \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") "
Jul 27 22:47:25.692557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692456    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfx7x\" (UniqueName: \"kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x\") pod \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\" (UID: \"12fbbfaa-5801-4ec5-b364-195b7698c1b4\") "
Jul 27 22:47:25.692557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692511    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rsfc\" (UniqueName: \"kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc\") pod \"eb4dd926-2fb1-4839-8485-c9436d30aeba\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") "
Jul 27 22:47:25.692557 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume\") pod \"eb4dd926-2fb1-4839-8485-c9436d30aeba\" (UID: \"eb4dd926-2fb1-4839-8485-c9436d30aeba\") "
Jul 27 22:47:25.692990 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692959    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12fbbfaa-5801-4ec5-b364-195b7698c1b4" (UID: "12fbbfaa-5801-4ec5-b364-195b7698c1b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:25.693097 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.692995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb4dd926-2fb1-4839-8485-c9436d30aeba" (UID: "eb4dd926-2fb1-4839-8485-c9436d30aeba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:25.694803 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.694782    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x" (OuterVolumeSpecName: "kube-api-access-bfx7x") pod "12fbbfaa-5801-4ec5-b364-195b7698c1b4" (UID: "12fbbfaa-5801-4ec5-b364-195b7698c1b4"). InnerVolumeSpecName "kube-api-access-bfx7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:25.694803 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.694790    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc" (OuterVolumeSpecName: "kube-api-access-4rsfc") pod "eb4dd926-2fb1-4839-8485-c9436d30aeba" (UID: "eb4dd926-2fb1-4839-8485-c9436d30aeba"). InnerVolumeSpecName "kube-api-access-4rsfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:25.793978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.793894    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rsfc\" (UniqueName: \"kubernetes.io/projected/eb4dd926-2fb1-4839-8485-c9436d30aeba-kube-api-access-4rsfc\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:25.793978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.793929    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4dd926-2fb1-4839-8485-c9436d30aeba-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:25.793978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.793939    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12fbbfaa-5801-4ec5-b364-195b7698c1b4-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:25.793978 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:25.793948    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfx7x\" (UniqueName: \"kubernetes.io/projected/12fbbfaa-5801-4ec5-b364-195b7698c1b4-kube-api-access-bfx7x\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:26.360505 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.360474    2578 generic.go:356] "Generic (PLEG): container finished" podID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" containerID="711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05" exitCode=0
Jul 27 22:47:26.360862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.360524    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"
Jul 27 22:47:26.360862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.360556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" event={"ID":"12fbbfaa-5801-4ec5-b364-195b7698c1b4","Type":"ContainerDied","Data":"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"}
Jul 27 22:47:26.360862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.360623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr" event={"ID":"12fbbfaa-5801-4ec5-b364-195b7698c1b4","Type":"ContainerDied","Data":"31344f33669660d05f4e3176897a990cd22ba5237ced9e386b8a224b29a0f06d"}
Jul 27 22:47:26.360862 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.360647    2578 scope.go:117] "RemoveContainer" containerID="711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"
Jul 27 22:47:26.362387 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.362363    2578 generic.go:356] "Generic (PLEG): container finished" podID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerID="2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e" exitCode=0
Jul 27 22:47:26.362572 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.362433    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"
Jul 27 22:47:26.362803 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.362446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" event={"ID":"eb4dd926-2fb1-4839-8485-c9436d30aeba","Type":"ContainerDied","Data":"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"}
Jul 27 22:47:26.362941 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.362838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x" event={"ID":"eb4dd926-2fb1-4839-8485-c9436d30aeba","Type":"ContainerDied","Data":"f306c0c026eff98636d184cabbdc1f59ac998005158545f095cb331e8fa7e44b"}
Jul 27 22:47:26.370001 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.369980    2578 scope.go:117] "RemoveContainer" containerID="711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"
Jul 27 22:47:26.370287 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:26.370270    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05\": container with ID starting with 711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05 not found: ID does not exist" containerID="711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"
Jul 27 22:47:26.370340 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.370296    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05"} err="failed to get container status \"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05\": rpc error: code = NotFound desc = could not find container \"711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05\": container with ID starting with 711a4101d6e429f4d5110ca3cdb72e820f5459527d3f8117a52ff18e855fdd05 not found: ID does not exist"
Jul 27 22:47:26.370340 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.370313    2578 scope.go:117] "RemoveContainer" containerID="2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"
Jul 27 22:47:26.378146 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.378124    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:47:26.378635 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.378620    2578 scope.go:117] "RemoveContainer" containerID="2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"
Jul 27 22:47:26.378899 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:26.378880    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e\": container with ID starting with 2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e not found: ID does not exist" containerID="2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"
Jul 27 22:47:26.378970 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.378906    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e"} err="failed to get container status \"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e\": rpc error: code = NotFound desc = could not find container \"2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e\": container with ID starting with 2502e796a2d286602b68894c810ccd92b07d3efd0675062c4580522f6cab5d4e not found: ID does not exist"
Jul 27 22:47:26.382500 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.382480    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c5cbe3-69ff459f78-hbglr"]
Jul 27 22:47:26.392426 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.392403    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:47:26.398140 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:26.398117    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c5cbe3-5d59cc454d-9hc2x"]
Jul 27 22:47:28.323216 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:28.323160    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" path="/var/lib/kubelet/pods/12fbbfaa-5801-4ec5-b364-195b7698c1b4/volumes"
Jul 27 22:47:28.323623 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:28.323553    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" path="/var/lib/kubelet/pods/eb4dd926-2fb1-4839-8485-c9436d30aeba/volumes"
Jul 27 22:47:28.341114 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:28.341079    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:47:29.720550 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:29.720513    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"]
Jul 27 22:47:29.726914 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:29.726885    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c5cbe3-7lcm2"]
Jul 27 22:47:30.324064 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:30.324028    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5141577e-9793-4f22-9a6c-9d89c124e2b6" path="/var/lib/kubelet/pods/5141577e-9793-4f22-9a6c-9d89c124e2b6/volumes"
Jul 27 22:47:36.855586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855553    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"]
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855944    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5141577e-9793-4f22-9a6c-9d89c124e2b6" containerName="revoke-keys"
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855956    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5141577e-9793-4f22-9a6c-9d89c124e2b6" containerName="revoke-keys"
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855965    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerName="payload-pre-processing"
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855970    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerName="payload-pre-processing"
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855987    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" containerName="payload-processing"
Jul 27 22:47:36.856009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.855993    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" containerName="payload-processing"
Jul 27 22:47:36.856227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.856059    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4dd926-2fb1-4839-8485-c9436d30aeba" containerName="payload-pre-processing"
Jul 27 22:47:36.856227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.856068    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="12fbbfaa-5801-4ec5-b364-195b7698c1b4" containerName="payload-processing"
Jul 27 22:47:36.856227 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.856076    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5141577e-9793-4f22-9a6c-9d89c124e2b6" containerName="revoke-keys"
Jul 27 22:47:36.859224 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.859208    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:36.862009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.861984    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k9rxw\""
Jul 27 22:47:36.862009 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.862000    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 22:47:36.866165 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.866138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"]
Jul 27 22:47:36.872982 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.872962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6\" (UID: \"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:36.974177 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.974146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6\" (UID: \"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:36.974797 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:36.974781    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6\" (UID: \"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:37.170618 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:37.170545    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:37.309188 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:37.309161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"]
Jul 27 22:47:37.311849 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:47:37.311821    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad3cfd3_c5b2_40b9_b09f_51cc3cfe6c1a.slice/crio-73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e WatchSource:0}: Error finding container 73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e: Status 404 returned error can't find the container with id 73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e
Jul 27 22:47:37.422457 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:37.422376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6" event={"ID":"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a","Type":"ContainerStarted","Data":"73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e"}
Jul 27 22:47:38.429124 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:38.429039    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" containerID="40761f96557b28c3d38cc58e09700ef3294b4f03ca6f7f69e1cf0fd5c333d639" exitCode=0
Jul 27 22:47:38.429496 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:38.429121    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6" event={"ID":"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a","Type":"ContainerDied","Data":"40761f96557b28c3d38cc58e09700ef3294b4f03ca6f7f69e1cf0fd5c333d639"}
Jul 27 22:47:39.572786 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:39.572766    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:39.608156 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:39.608124    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca\") pod \"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a\" (UID: \"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a\") "
Jul 27 22:47:39.612877 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:39.612846    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" (UID: "0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:39.709768 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:39.709627    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a-maas-api-service-ca\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:40.442617 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:40.442593    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"
Jul 27 22:47:40.442790 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:40.442592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6" event={"ID":"0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a","Type":"ContainerDied","Data":"73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e"}
Jul 27 22:47:40.442790 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:40.442730    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73852a08ee1f258cb421c527de9865fd5a65f8a5e23c3b89cfedeb34b1a0d56e"
Jul 27 22:47:47.418602 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.418561    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:47:47.419237 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.418813    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" podUID="b9348ed3-7f9c-4364-a307-2adcc09f135b" containerName="payload-processing" containerID="cri-o://685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2" gracePeriod=30
Jul 27 22:47:47.436726 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.436696    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:47:47.436948 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.436924    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" podUID="8ee62c1a-c753-459f-96a2-b06a04cb358d" containerName="payload-pre-processing" containerID="cri-o://fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e" gracePeriod=30
Jul 27 22:47:47.697931 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.697905    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:47:47.719985 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.719961    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:47:47.781337 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8fqf\" (UniqueName: \"kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf\") pod \"8ee62c1a-c753-459f-96a2-b06a04cb358d\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") "
Jul 27 22:47:47.781536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781357    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume\") pod \"8ee62c1a-c753-459f-96a2-b06a04cb358d\" (UID: \"8ee62c1a-c753-459f-96a2-b06a04cb358d\") "
Jul 27 22:47:47.781536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781426    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgwxd\" (UniqueName: \"kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd\") pod \"b9348ed3-7f9c-4364-a307-2adcc09f135b\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") "
Jul 27 22:47:47.781536 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781451    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume\") pod \"b9348ed3-7f9c-4364-a307-2adcc09f135b\" (UID: \"b9348ed3-7f9c-4364-a307-2adcc09f135b\") "
Jul 27 22:47:47.781805 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781783    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ee62c1a-c753-459f-96a2-b06a04cb358d" (UID: "8ee62c1a-c753-459f-96a2-b06a04cb358d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:47.781874 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.781853    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9348ed3-7f9c-4364-a307-2adcc09f135b" (UID: "b9348ed3-7f9c-4364-a307-2adcc09f135b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 22:47:47.783605 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.783579    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd" (OuterVolumeSpecName: "kube-api-access-cgwxd") pod "b9348ed3-7f9c-4364-a307-2adcc09f135b" (UID: "b9348ed3-7f9c-4364-a307-2adcc09f135b"). InnerVolumeSpecName "kube-api-access-cgwxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:47.783766 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.783737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf" (OuterVolumeSpecName: "kube-api-access-w8fqf") pod "8ee62c1a-c753-459f-96a2-b06a04cb358d" (UID: "8ee62c1a-c753-459f-96a2-b06a04cb358d"). InnerVolumeSpecName "kube-api-access-w8fqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 22:47:47.883160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.883119    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8fqf\" (UniqueName: \"kubernetes.io/projected/8ee62c1a-c753-459f-96a2-b06a04cb358d-kube-api-access-w8fqf\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:47.883160 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.883160    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee62c1a-c753-459f-96a2-b06a04cb358d-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:47.883365 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.883175    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgwxd\" (UniqueName: \"kubernetes.io/projected/b9348ed3-7f9c-4364-a307-2adcc09f135b-kube-api-access-cgwxd\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:47.883365 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:47.883188    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9348ed3-7f9c-4364-a307-2adcc09f135b-plugins-config-volume\") on node \"ip-10-0-136-91.ec2.internal\" DevicePath \"\""
Jul 27 22:47:48.486080 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.486047    2578 generic.go:356] "Generic (PLEG): container finished" podID="8ee62c1a-c753-459f-96a2-b06a04cb358d" containerID="fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e" exitCode=0
Jul 27 22:47:48.486543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.486096    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"
Jul 27 22:47:48.486543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.486139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" event={"ID":"8ee62c1a-c753-459f-96a2-b06a04cb358d","Type":"ContainerDied","Data":"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"}
Jul 27 22:47:48.486543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.486190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn" event={"ID":"8ee62c1a-c753-459f-96a2-b06a04cb358d","Type":"ContainerDied","Data":"8d0bcf5943c9ba0bc0f95435465397e40504c253ad1e156e3e63a843efec7d69"}
Jul 27 22:47:48.486543 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.486219    2578 scope.go:117] "RemoveContainer" containerID="fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"
Jul 27 22:47:48.487769 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.487720    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9348ed3-7f9c-4364-a307-2adcc09f135b" containerID="685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2" exitCode=0
Jul 27 22:47:48.487835 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.487766    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"
Jul 27 22:47:48.487835 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.487798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" event={"ID":"b9348ed3-7f9c-4364-a307-2adcc09f135b","Type":"ContainerDied","Data":"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"}
Jul 27 22:47:48.487941 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.487848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz" event={"ID":"b9348ed3-7f9c-4364-a307-2adcc09f135b","Type":"ContainerDied","Data":"572cb60bf1022ef5c8c8f6e0a77253819346a2390482a64e301e3eb488d03ee2"}
Jul 27 22:47:48.495586 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.495568    2578 scope.go:117] "RemoveContainer" containerID="fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"
Jul 27 22:47:48.495878 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:48.495857    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e\": container with ID starting with fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e not found: ID does not exist" containerID="fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"
Jul 27 22:47:48.495939 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.495889    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e"} err="failed to get container status \"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e\": rpc error: code = NotFound desc = could not find container \"fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e\": container with ID starting with fc93f33a88cdd9768759937048a5365ec4a7080691b65735c2830a28cdc41e5e not found: ID does not exist"
Jul 27 22:47:48.495939 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.495910    2578 scope.go:117] "RemoveContainer" containerID="685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"
Jul 27 22:47:48.502990 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.502964    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:47:48.504096 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.504084    2578 scope.go:117] "RemoveContainer" containerID="685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"
Jul 27 22:47:48.505051 ip-10-0-136-91 kubenswrapper[2578]: E0727 22:47:48.504685    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2\": container with ID starting with 685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2 not found: ID does not exist" containerID="685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"
Jul 27 22:47:48.505051 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.504794    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2"} err="failed to get container status \"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2\": rpc error: code = NotFound desc = could not find container \"685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2\": container with ID starting with 685b9c84351b73f9fd129779aff1b8960034321ec43834084795799626905be2 not found: ID does not exist"
Jul 27 22:47:48.514083 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.514055    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f36b41-6c65b5bdf4-rbqlz"]
Jul 27 22:47:48.525109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.525089    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:47:48.528982 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:48.528960    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f36b41-7f9cdd497b-5mrnn"]
Jul 27 22:47:50.323284 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:50.323246    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee62c1a-c753-459f-96a2-b06a04cb358d" path="/var/lib/kubelet/pods/8ee62c1a-c753-459f-96a2-b06a04cb358d/volumes"
Jul 27 22:47:50.323798 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:50.323773    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9348ed3-7f9c-4364-a307-2adcc09f135b" path="/var/lib/kubelet/pods/b9348ed3-7f9c-4364-a307-2adcc09f135b/volumes"
Jul 27 22:47:50.341435 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:50.341411    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-86dbt"]
Jul 27 22:47:51.932957 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:51.932925    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"]
Jul 27 22:47:51.935442 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:51.935412    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f36b41-bc4h6"]
Jul 27 22:47:52.322703 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:47:52.322654    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" path="/var/lib/kubelet/pods/0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a/volumes"
Jul 27 22:48:04.580974 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:04.580941    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z4rtz_01aaad7e-1e22-46b0-9ade-fa5bdc9fca03/manager/0.log"
Jul 27 22:48:04.697135 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:04.697099    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gqvvs_f0922773-e30f-41fd-a880-625e4617cce8/manager/0.log"
Jul 27 22:48:05.424583 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:05.424552    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8fb7b654-rp2mj_c3737160-6d9d-4577-b5ff-2d820e854902/copy-manifests/0.log"
Jul 27 22:48:05.432140 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:05.432116    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8fb7b654-rp2mj_c3737160-6d9d-4577-b5ff-2d820e854902/manager/0.log"
Jul 27 22:48:07.522346 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:07.522311    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zw628_6538384e-ba9a-477f-86e9-abca7b938934/manager/0.log"
Jul 27 22:48:07.630286 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:07.630250    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s7rnr_357f03da-b554-45dd-aea1-79aafe5c7845/manager/0.log"
Jul 27 22:48:07.734811 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:07.734783    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j5qjx_1656b79b-ae4e-492c-b6a2-8af32ef633df/kuadrant-console-plugin/0.log"
Jul 27 22:48:07.855118 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:07.855030    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d94sm_38284300-bbf6-4875-8cf5-f93a0ac7eba7/registry-server/0.log"
Jul 27 22:48:07.972159 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:07.972127    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vc2kg_b92ce823-3263-4c5f-8b84-71c33ce922a2/manager/0.log"
Jul 27 22:48:08.083061 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:08.083022    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-86dbt_8dab191a-5547-44e2-a9d4-687d831ee5e7/limitador/0.log"
Jul 27 22:48:08.204120 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:08.204009    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6ntp7_b38c447c-a4a5-4ba8-a324-3b8582e4d29c/manager/0.log"
Jul 27 22:48:08.781028 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:08.780992    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fcc8648dc-hwn9f_7e5f098d-b052-40cf-9bec-7ffd1f7d1191/kube-auth-proxy/0.log"
Jul 27 22:48:09.777690 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:09.777645    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-72lvt_d7a2bd50-5898-42ef-8dc8-825d3daad16d/main/0.log"
Jul 27 22:48:10.112443 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:10.112347    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4vlx9_a67bbc9d-22e2-48b2-9275-44d3171921f1/main/0.log"
Jul 27 22:48:18.604082 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:18.604052    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qlxss_e6d161cf-b051-4a5c-98f0-ac23f474b98b/global-pull-secret-syncer/0.log"
Jul 27 22:48:18.711356 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:18.711323    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c2nbr_ef7d343e-8673-4b23-affa-63f4b78fb424/konnectivity-agent/0.log"
Jul 27 22:48:18.806815 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:18.806785    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-91.ec2.internal_cabb2983197905ab0fef18f09916296b/haproxy/0.log"
Jul 27 22:48:23.892958 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:23.892921    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zw628_6538384e-ba9a-477f-86e9-abca7b938934/manager/0.log"
Jul 27 22:48:23.915611 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:23.915579    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-s7rnr_357f03da-b554-45dd-aea1-79aafe5c7845/manager/0.log"
Jul 27 22:48:23.941562 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:23.941534    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j5qjx_1656b79b-ae4e-492c-b6a2-8af32ef633df/kuadrant-console-plugin/0.log"
Jul 27 22:48:24.009824 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:24.009791    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-d94sm_38284300-bbf6-4875-8cf5-f93a0ac7eba7/registry-server/0.log"
Jul 27 22:48:24.250264 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:24.250231    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vc2kg_b92ce823-3263-4c5f-8b84-71c33ce922a2/manager/0.log"
Jul 27 22:48:24.282826 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:24.282803    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-86dbt_8dab191a-5547-44e2-a9d4-687d831ee5e7/limitador/0.log"
Jul 27 22:48:24.405076 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:24.405046    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-6ntp7_b38c447c-a4a5-4ba8-a324-3b8582e4d29c/manager/0.log"
Jul 27 22:48:26.541490 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:26.541463    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-rls2f_e76c5bfe-5735-4b0a-8c4b-2070e2f0587a/monitoring-plugin/0.log"
Jul 27 22:48:26.577751 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:26.577726    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68z7n_4f6294bc-568c-4ed9-9a61-7be11bbb1e7d/node-exporter/0.log"
Jul 27 22:48:26.603432 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:26.603408    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68z7n_4f6294bc-568c-4ed9-9a61-7be11bbb1e7d/kube-rbac-proxy/0.log"
Jul 27 22:48:26.624279 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:26.624257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68z7n_4f6294bc-568c-4ed9-9a61-7be11bbb1e7d/init-textfile/0.log"
Jul 27 22:48:27.053185 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:27.053151    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-r75c7_2c42892f-7c7b-410e-8890-c95e64da3d8d/prometheus-operator/0.log"
Jul 27 22:48:27.069896 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:27.069866    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-r75c7_2c42892f-7c7b-410e-8890-c95e64da3d8d/kube-rbac-proxy/0.log"
Jul 27 22:48:27.125055 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:27.125026    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5ff4b67-bwt52_c54c377e-825b-4421-a9ba-3f500c424040/telemeter-client/0.log"
Jul 27 22:48:27.146552 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:27.146524    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5ff4b67-bwt52_c54c377e-825b-4421-a9ba-3f500c424040/reload/0.log"
Jul 27 22:48:27.169989 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:27.169963    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cf5ff4b67-bwt52_c54c377e-825b-4421-a9ba-3f500c424040/kube-rbac-proxy/0.log"
Jul 27 22:48:28.578907 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.578859    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"]
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579242    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" containerName="revoke-keys"
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579254    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" containerName="revoke-keys"
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579273    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee62c1a-c753-459f-96a2-b06a04cb358d" containerName="payload-pre-processing"
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579278    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee62c1a-c753-459f-96a2-b06a04cb358d" containerName="payload-pre-processing"
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579284    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9348ed3-7f9c-4364-a307-2adcc09f135b" containerName="payload-processing"
Jul 27 22:48:28.579297 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579290    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9348ed3-7f9c-4364-a307-2adcc09f135b" containerName="payload-processing"
Jul 27 22:48:28.579492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579353    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad3cfd3-c5b2-40b9-b09f-51cc3cfe6c1a" containerName="revoke-keys"
Jul 27 22:48:28.579492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579360    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9348ed3-7f9c-4364-a307-2adcc09f135b" containerName="payload-processing"
Jul 27 22:48:28.579492 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.579369    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee62c1a-c753-459f-96a2-b06a04cb358d" containerName="payload-pre-processing"
Jul 27 22:48:28.582617 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.582602    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.585554 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.585531    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j945x\"/\"openshift-service-ca.crt\""
Jul 27 22:48:28.586850 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.586834    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j945x\"/\"default-dockercfg-94sd4\""
Jul 27 22:48:28.586920 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.586867    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j945x\"/\"kube-root-ca.crt\""
Jul 27 22:48:28.594153 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.594133    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"]
Jul 27 22:48:28.631879 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.631848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-lib-modules\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.632049 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.631888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-proc\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.632049 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.631908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvlcz\" (UniqueName: \"kubernetes.io/projected/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-kube-api-access-lvlcz\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.632049 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.631961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-podres\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.632171 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.632049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-sys\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732532 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-podres\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-sys\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-lib-modules\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-proc\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvlcz\" (UniqueName: \"kubernetes.io/projected/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-kube-api-access-lvlcz\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-sys\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732707    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-proc\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.732743 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-podres\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.733000 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.732779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-lib-modules\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.741283 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.741258    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvlcz\" (UniqueName: \"kubernetes.io/projected/b41e3c6a-6184-4001-a1d2-5b9de0c376cd-kube-api-access-lvlcz\") pod \"perf-node-gather-daemonset-vrzs5\" (UID: \"b41e3c6a-6184-4001-a1d2-5b9de0c376cd\") " pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:28.893639 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:28.893535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:29.031988 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.031963    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"]
Jul 27 22:48:29.033966 ip-10-0-136-91 kubenswrapper[2578]: W0727 22:48:29.033941    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb41e3c6a_6184_4001_a1d2_5b9de0c376cd.slice/crio-8028c6d7f7488c396dc30b87b0720083d2b39fa3eadcb83e1f94de7f4704b825 WatchSource:0}: Error finding container 8028c6d7f7488c396dc30b87b0720083d2b39fa3eadcb83e1f94de7f4704b825: Status 404 returned error can't find the container with id 8028c6d7f7488c396dc30b87b0720083d2b39fa3eadcb83e1f94de7f4704b825
Jul 27 22:48:29.173344 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.173258    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/2.log"
Jul 27 22:48:29.180802 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.180781    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-zmwg9_8db60bac-31d7-481a-9f6a-53789dc1be2f/console-operator/3.log"
Jul 27 22:48:29.662454 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.662420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54744bbcb4-8jhp8_05f20452-1e79-4faa-a040-3174da6d14e0/console/0.log"
Jul 27 22:48:29.704384 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.704355    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-nnmzd_0393ec85-e28b-4f36-b3e6-3ecf99f8c619/download-server/0.log"
Jul 27 22:48:29.708129 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.708103    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5" event={"ID":"b41e3c6a-6184-4001-a1d2-5b9de0c376cd","Type":"ContainerStarted","Data":"ce2a48f67db9b80cb690ff1c0dcedc33809a0355c27cedc50e985928132a14fa"}
Jul 27 22:48:29.708271 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.708159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5" event={"ID":"b41e3c6a-6184-4001-a1d2-5b9de0c376cd","Type":"ContainerStarted","Data":"8028c6d7f7488c396dc30b87b0720083d2b39fa3eadcb83e1f94de7f4704b825"}
Jul 27 22:48:29.708271 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.708185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:29.723501 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:29.723455    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5" podStartSLOduration=1.723437166 podStartE2EDuration="1.723437166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 22:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 22:48:29.722840483 +0000 UTC m=+3219.909681142" watchObservedRunningTime="2026-07-27 22:48:29.723437166 +0000 UTC m=+3219.910277826"
Jul 27 22:48:31.374224 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:31.374193    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzglt_652a8763-7cbf-4c08-abdb-0faabcaf7d28/dns/0.log"
Jul 27 22:48:31.395856 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:31.395828    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wzglt_652a8763-7cbf-4c08-abdb-0faabcaf7d28/kube-rbac-proxy/0.log"
Jul 27 22:48:31.474829 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:31.474805    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pxm6b_9f091a7e-c114-4d02-968a-9cd9f8f404eb/dns-node-resolver/0.log"
Jul 27 22:48:32.011953 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:32.011905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54cc4f975-ccxw2_950c2c66-ead4-460e-a1d1-be6d1193e09b/registry/0.log"
Jul 27 22:48:32.037366 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:32.037338    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6v88q_75184c67-8fdc-4da3-9aa8-97dcb9f948d8/node-ca/0.log"
Jul 27 22:48:33.308306 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:33.308269    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fcc8648dc-hwn9f_7e5f098d-b052-40cf-9bec-7ffd1f7d1191/kube-auth-proxy/0.log"
Jul 27 22:48:34.295758 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:34.295730    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-967mv_e666f0d7-2854-46f3-b9db-748d7489971e/serve-healthcheck-canary/0.log"
Jul 27 22:48:34.854844 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:34.854814    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-5rpfw_eda4dc46-cd71-49b2-a7d1-1915a6a86610/insights-operator/0.log"
Jul 27 22:48:34.857768 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:34.857732    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-5rpfw_eda4dc46-cd71-49b2-a7d1-1915a6a86610/insights-operator/1.log"
Jul 27 22:48:35.026012 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:35.025985    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpnlq_7e7ad512-7204-4cbc-9c3d-a8054d155126/kube-rbac-proxy/0.log"
Jul 27 22:48:35.049558 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:35.049534    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpnlq_7e7ad512-7204-4cbc-9c3d-a8054d155126/exporter/0.log"
Jul 27 22:48:35.073747 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:35.073723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpnlq_7e7ad512-7204-4cbc-9c3d-a8054d155126/extractor/0.log"
Jul 27 22:48:35.723287 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:35.723259    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j945x/perf-node-gather-daemonset-vrzs5"
Jul 27 22:48:37.149566 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:37.149536    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z4rtz_01aaad7e-1e22-46b0-9ade-fa5bdc9fca03/manager/0.log"
Jul 27 22:48:37.547509 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:37.547469    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gqvvs_f0922773-e30f-41fd-a880-625e4617cce8/manager/0.log"
Jul 27 22:48:38.141658 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:38.141611    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8fb7b654-rp2mj_c3737160-6d9d-4577-b5ff-2d820e854902/manager/0.log"
Jul 27 22:48:38.165880 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:38.165849    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8fb7b654-rp2mj_c3737160-6d9d-4577-b5ff-2d820e854902/copy-manifests/0.log"
Jul 27 22:48:38.807368 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:38.807342    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fcc8648dc-hwn9f_7e5f098d-b052-40cf-9bec-7ffd1f7d1191/kube-auth-proxy/0.log"
Jul 27 22:48:40.755343 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:40.755316    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fcc8648dc-hwn9f_7e5f098d-b052-40cf-9bec-7ffd1f7d1191/kube-auth-proxy/0.log"
Jul 27 22:48:44.913177 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:44.913140    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7_3d7130e9-1bcb-4489-86eb-2abfe7f9479a/kube-storage-version-migrator-operator/1.log"
Jul 27 22:48:44.915045 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:44.915003    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rkkd7_3d7130e9-1bcb-4489-86eb-2abfe7f9479a/kube-storage-version-migrator-operator/0.log"
Jul 27 22:48:46.375873 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.375844    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/kube-multus-additional-cni-plugins/0.log"
Jul 27 22:48:46.400109 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.400082    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/egress-router-binary-copy/0.log"
Jul 27 22:48:46.422070 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.422044    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/cni-plugins/0.log"
Jul 27 22:48:46.446800 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.446777    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/bond-cni-plugin/0.log"
Jul 27 22:48:46.472244 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.472220    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/routeoverride-cni/0.log"
Jul 27 22:48:46.494323 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.494294    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/whereabouts-cni-bincopy/0.log"
Jul 27 22:48:46.516133 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.516109    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sc5mp_4fe713f4-7663-4694-83df-feab1b0a9202/whereabouts-cni/0.log"
Jul 27 22:48:46.605247 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.605217    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qgxl7_43159b45-826d-476f-9b4c-694e11a72cef/kube-multus/0.log"
Jul 27 22:48:46.631102 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.631004    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4xm4r_d8edead7-ce5c-4c47-80d1-bf5cb9d53525/network-metrics-daemon/0.log"
Jul 27 22:48:46.648048 ip-10-0-136-91 kubenswrapper[2578]: I0727 22:48:46.648026    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4xm4r_d8edead7-ce5c-4c47-80d1-bf5cb9d53525/kube-rbac-proxy/0.log"