Jul 28 20:23:00.995042 ip-10-0-139-131 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 20:23:00.995052 ip-10-0-139-131 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 20:23:00.995059 ip-10-0-139-131 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 20:23:00.995292 ip-10-0-139-131 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 20:23:11.153150 ip-10-0-139-131 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 20:23:11.153173 ip-10-0-139-131 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot af9545d3618340fa8618b0eb546f61b2 --
Jul 28 20:25:04.706017 ip-10-0-139-131 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 20:25:05.054323 ip-10-0-139-131 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:25:05.054323 ip-10-0-139-131 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 20:25:05.054323 ip-10-0-139-131 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:25:05.054323 ip-10-0-139-131 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 20:25:05.054323 ip-10-0-139-131 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 20:25:05.056252 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.056130    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061635    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061656    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061661    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061665    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061668    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.061661 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061671    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061674    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061677    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061679    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061682    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061685    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061687    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061690    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061693    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061696    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061699    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061702    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061704    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061707    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061710    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061712    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061715    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061718    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061720    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061723    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.061898 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061725    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061728    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061730    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061733    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061735    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061741    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061744    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061747    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061749    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061752    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061754    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061757    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061759    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061761    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061765    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061768    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061771    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061773    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061776    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061779    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.062445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061781    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061784    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061786    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061789    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061791    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061795    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061798    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061801    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061804    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061807    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061809    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061812    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061814    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061817    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061820    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061822    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061825    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061827    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061830    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061833    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.062943 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061835    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061838    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061842    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061846    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061849    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061852    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061856    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061860    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061863    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061866    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061870    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061873    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061876    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061879    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061881    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061884    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061886    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061889    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061892    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.063445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061895    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061899    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061901    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061904    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061907    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061909    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061912    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061915    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061918    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061920    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061923    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061925    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061928    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061930    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.061933    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063472    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063481    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063484    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063487    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063490    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.063905 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063492    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063495    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063498    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063500    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063503    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063505    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063508    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063511    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063513    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063515    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063518    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063521    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063524    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063527    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063531    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063533    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063536    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063539    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063541    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063545    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.064417 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063547    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063550    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063552    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063555    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063558    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063560    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063563    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063565    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063569    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063571    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063574    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063577    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063579    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063582    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063585    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063587    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063590    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063592    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063595    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063598    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.065053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063600    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063603    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063605    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063608    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063611    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063615    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063620    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063624    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063627    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063645    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063648    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063652    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063655    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063657    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063661    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063664    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063667    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063670    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063672    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.065588 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063675    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063678    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063681    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063683    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063686    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063690    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063692    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063695    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063699    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063704    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063708    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063712    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063715    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063718    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063721    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063724    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063727    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063730    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063733    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.066081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063737    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063741    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063743    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063746    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063748    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063751    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063754    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063756    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063759    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063762    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063764    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063767    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063770    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063772    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063775    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.063777    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064172    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064178    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064182    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.066564 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064185    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064188    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064191    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064193    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064196    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064199    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064201    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064204    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064206    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064209    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064212    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064214    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064218    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064222    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064225    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064228    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064230    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064233    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064236    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064239    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064241    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.067051 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064244    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064246    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064249    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064251    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064254    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064256    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064259    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064262    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064266    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064269    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064272    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064275    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064278    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064281    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064283    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064286    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064290    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064293    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064295    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.067616 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064298    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064301    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064304    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064307    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064309    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064312    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064316    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064319    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064322    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064324    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064327    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064329    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064332    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064336    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064340    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064343    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064345    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064349    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064353    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.068080 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064356    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064359    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064379    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064383    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064386    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064388    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064391    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064393    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064396    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064399    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064402    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064404    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064407    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064410    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064413    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064415    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064418    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064421    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064423    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064426    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.068557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064432    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064435    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064438    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064440    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064443    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064445    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064448    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064451    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064453    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064456    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064458    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064461    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064463    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064466    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064469    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064472    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.064474    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064542    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064550    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064558    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064562    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 20:25:05.069081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064566    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064570    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064574    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064579    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064582    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064585    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064588    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064592    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064598    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064601    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064604    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064606    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064610    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064613    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064617    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064621    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064624    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064627    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064631    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064634    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064637    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064640    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064644    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064647    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 20:25:05.069604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064650    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064653    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064656    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064659    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064663    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064666    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064669    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064672    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064675    2576 flags.go:64] FLAG: --enable-server="true"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064679    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064684    2576 flags.go:64] FLAG: --event-burst="100"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064688    2576 flags.go:64] FLAG: --event-qps="50"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064690    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064694    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064697    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064701    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064704    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064707    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064710    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064713    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064716    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064719    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064722    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064725    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064728    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 20:25:05.070181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064731    2576 flags.go:64] FLAG: --feature-gates=""
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064735    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064738    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064741    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064744    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064748    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064750    2576 flags.go:64] FLAG: --help="false"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064753    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064757    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064759    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064762    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064766    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064769    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064773    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064775    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064778    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064781    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064784    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064788    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064791    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064794    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064797    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064800    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064803    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 20:25:05.070827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064806    2576 flags.go:64] FLAG: --lock-file=""
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064809    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064811    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064814    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064820    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064824    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064827    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064830    2576 flags.go:64] FLAG: --logging-format="text"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064832    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064836    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064838    2576 flags.go:64] FLAG: --manifest-url=""
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064841    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064846    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064849    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064853    2576 flags.go:64] FLAG: --max-pods="110"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064856    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064859    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064861    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064864    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064867    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064870    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064874    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064881    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064884    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064887    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 20:25:05.071467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064890    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064893    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064899    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064902    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064905    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064908    2576 flags.go:64] FLAG: --port="10250"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064911    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064914    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d5b32a7951e7204"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064917    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064920    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064923    2576 flags.go:64] FLAG: --register-node="true"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064926    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064930    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064933    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064936    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064939    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064942    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064945    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064948    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064952    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064959    2576 flags.go:64] FLAG: --runonce="false"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064962    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064965    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064968    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064971    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 20:25:05.072083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064974    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064977    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064981    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064984    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064987    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064990    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064993    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.064996    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065000    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065003    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065006    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065011    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065014    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065017    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065021    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065024    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065027    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065030    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065033    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065036    2576 flags.go:64] FLAG: --v="2"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065040    2576 flags.go:64] FLAG: --version="false"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065049    2576 flags.go:64] FLAG: --vmodule=""
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065058    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.065061    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065193    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.072750 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065198    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065201    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065206    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065209    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065212    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065215    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065217    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065220    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065223    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065226    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065228    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065231    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065234    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065239    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065242    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065244    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065247    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065249    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065252    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065255    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.073379 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065257    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065260    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065262    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065265    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065267    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065270    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065272    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065275    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065277    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065280    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065283    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065285    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065288    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065290    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065295    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065297    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065301    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065305    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065308    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065311    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.073883 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065314    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065317    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065319    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065322    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065324    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065328    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065331    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065333    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065336    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065339    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065341    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065344    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065347    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065349    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065352    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065354    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065357    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065373    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065379    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065382    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.074375 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065385    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065388    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065391    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065394    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065397    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065399    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065403    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065406    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065410    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065413    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065416    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065419    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065422    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065425    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065428    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065430    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065433    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065437    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065440    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.074882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065444    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065447    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065451    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065453    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065456    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065459    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065463    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065465    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065468    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065471    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065474    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065476    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065479    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065481    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065484    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065486    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065490    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065493    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.075344 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.065496    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.066070    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.073650    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.073673    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.073695    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.073704    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073769    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073775    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073779    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073782    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073785    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073788    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073790    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073794    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073797    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.075809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073799    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073802    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073805    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073808    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073810    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073813    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073815    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073818    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073821    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073825    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073828    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073831    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073833    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073836    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073838    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073841    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073843    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073846    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073848    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073851    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.076184 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073861    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073864    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073866    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073869    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073871    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073875    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073879    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073882    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073885    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073888    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073892    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073894    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073897    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073900    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073902    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073905    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073907    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073911    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073914    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.076713 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073918    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073920    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073923    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073926    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073929    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073932    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073935    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073937    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073940    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073942    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073945    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073947    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073949    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073952    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073961    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073963    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073966    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073969    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073971    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073974    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.077190 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073976    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073979    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073981    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073984    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073987    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073989    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073992    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073994    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073997    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.073999    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074002    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074004    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074007    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074009    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074012    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074014    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074017    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074020    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074022    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074025    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.077697 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074027    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074030    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074032    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074035    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074038    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074040    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074043    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074051    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074054    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074056    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074059    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.074064    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074215    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074220    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074223    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 20:25:05.078186 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074226    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074229    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074232    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074235    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074237    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074240    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074242    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074245    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074247    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074249    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074252    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074255    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074257    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074260    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074264    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074268    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074271    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074273    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074276    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 20:25:05.078580 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074279    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074282    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074284    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074286    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074289    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074292    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074300    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074303    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074306    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074308    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074311    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074314    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074316    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074319    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074321    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074324    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074326    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074329    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074331    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074334    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 20:25:05.079059 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074337    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074339    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074342    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074344    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074347    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074349    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074352    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074355    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074358    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074373    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074376    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074379    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074382    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074385    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074388    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074391    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074393    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074397    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074400    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074410    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 20:25:05.079563 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074413    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074416    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074418    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074421    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074423    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074426    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074429    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074431    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074434    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074437    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074439    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074442    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074445    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074448    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074451    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074455    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074457    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074460    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074462    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 20:25:05.080053 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074465    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074468    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074471    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074475    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074477    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074480    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074483    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074486    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074488    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074492    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074494    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074497    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074500    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074508    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074510    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074513    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074515    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 20:25:05.080537 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:05.074518    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.074523    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.074673    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.076959    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.078322    2576 server.go:1019] "Starting client certificate rotation"
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.078424    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.078461    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 20:25:05.080949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.080636    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:25:05.084854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.084825    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 20:25:05.095789 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.095768    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 28 20:25:05.098164 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.098145    2576 log.go:25] "Validated CRI v1 image API"
Jul 28 20:25:05.100620 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.100602    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 20:25:05.103243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.103217    2576 fs.go:135] Filesystem UUIDs: map[00563914-4b01-4907-8e36-b6850d30b048:/dev/nvme0n1p3 514270c0-0fc6-4cf6-afab-b6de7f806991:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 28 20:25:05.103305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.103243    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 20:25:05.108404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.108383    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:25:05.108881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.108765    2576 manager.go:216] Machine: {Timestamp:2026-07-28 20:25:05.107392981 +0000 UTC m=+0.292497678 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102305 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22ab78a2149678257c3e071e876715 SystemUUID:ec22ab78-a214-9678-257c-3e071e876715 BootID:af9545d3-6183-40fa-8618-b0eb546f61b2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:2c:5e:7a:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:2c:5e:7a:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:55:b9:98:3a:1a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 20:25:05.108881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.108882    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 20:25:05.109038 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.109027    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 20:25:05.112188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.112160    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 20:25:05.112340 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.112190    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-131.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 20:25:05.112810 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.112799    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 20:25:05.112847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.112813    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 20:25:05.112847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.112821    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:25:05.113451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.113441    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 20:25:05.115167 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.115155    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.118782    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.118921    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.118962    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.118975    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.118992    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.119027    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 28 20:25:05.119202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.119037    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 20:25:05.119531 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.119494    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:25:05.119531 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.119531    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 20:25:05.121749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.121729    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 20:25:05.123713 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.123697    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 20:25:05.123762 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.123733    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 20:25:05.124495 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124479    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 20:25:05.124539 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124501    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 20:25:05.124539 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124509    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 20:25:05.124539 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124515    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 20:25:05.124539 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124524    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 20:25:05.124539 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124531    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124548    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124555    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124561    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124568    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124581    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 20:25:05.124683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.124586    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 20:25:05.126041 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.126027    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 20:25:05.126041 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.126041    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 20:25:05.129702 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.129687    2576 server.go:1268] "Started kubelet"
Jul 28 20:25:05.129788 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.129741    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 20:25:05.129882 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.129824    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 20:25:05.129939 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.129911    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 20:25:05.130733 ip-10-0-139-131 systemd[1]: Started Kubernetes Kubelet.
Jul 28 20:25:05.134511 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.134486    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-131.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 20:25:05.134688 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.134660    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 20:25:05.135036 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.135002    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 20:25:05.135138 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.135122    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 20:25:05.136159 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.136137    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 20:25:05.137044 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.137029    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 20:25:05.141457 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.140578    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-131.ec2.internal.18c68de23d4e60ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-131.ec2.internal,UID:ip-10-0-139-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-131.ec2.internal,},FirstTimestamp:2026-07-28 20:25:05.129652462 +0000 UTC m=+0.314757219,LastTimestamp:2026-07-28 20:25:05.129652462 +0000 UTC m=+0.314757219,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-131.ec2.internal,}"
Jul 28 20:25:05.142869 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.142855    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 20:25:05.142922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.142870    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 20:25:05.143240 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.143023    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.143575 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143556    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 20:25:05.143749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143609    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 20:25:05.143875 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143752    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 20:25:05.143875 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143753    2576 factory.go:55] Registering systemd factory
Jul 28 20:25:05.143875 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143793    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 28 20:25:05.144018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143889    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 20:25:05.144018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.143900    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 20:25:05.144018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144002    2576 factory.go:153] Registering CRI-O factory
Jul 28 20:25:05.144018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144017    2576 factory.go:223] Registration of the crio container factory successfully
Jul 28 20:25:05.144203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144076    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 20:25:05.144203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144100    2576 factory.go:103] Registering Raw factory
Jul 28 20:25:05.144203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144113    2576 manager.go:1196] Started watching for new ooms in manager
Jul 28 20:25:05.144648 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.144631    2576 manager.go:349] Starting recovery of all containers
Jul 28 20:25:05.149831 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.149805    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 20:25:05.149947 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.149835    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 20:25:05.151986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.151962    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnkm6"
Jul 28 20:25:05.155852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.155830    2576 manager.go:354] Recovery completed
Jul 28 20:25:05.157535 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.157401    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 20:25:05.161222 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.161208    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.162776 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.162756    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnkm6"
Jul 28 20:25:05.163337 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163320    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.163424 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163359    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.163424 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163387    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.163955 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163940    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 20:25:05.163955 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163953    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 20:25:05.164055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163961    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.164055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.163968    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 20:25:05.164821 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.164803    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.164899 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.164838    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.164899 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.164852    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.165484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.165471    2576 policy_none.go:49] "None policy: Start"
Jul 28 20:25:05.165526 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.165488    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 20:25:05.165526 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.165500    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 20:25:05.165711 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.165633    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-131.ec2.internal.18c68de23f506871  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-131.ec2.internal,UID:ip-10-0-139-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-131.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-131.ec2.internal,},FirstTimestamp:2026-07-28 20:25:05.163339889 +0000 UTC m=+0.348444600,LastTimestamp:2026-07-28 20:25:05.163339889 +0000 UTC m=+0.348444600,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-131.ec2.internal,}"
Jul 28 20:25:05.166709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.166697    2576 policy_none.go:47] "Start"
Jul 28 20:25:05.212391 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.212152    2576 manager.go:337] "Starting Device Plugin manager"
Jul 28 20:25:05.212518 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.212453    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 20:25:05.212518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.212469    2576 server.go:87] "Starting device plugin registration server"
Jul 28 20:25:05.212800 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.212787    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 20:25:05.212835 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.212804    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 20:25:05.212963 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.212929    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 20:25:05.213055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.213023    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 20:25:05.213055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.213034    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 20:25:05.213731 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.213705    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 20:25:05.213852 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.213764    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.292430 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.292389    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 20:25:05.293777 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.293760    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 20:25:05.293843 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.293801    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 20:25:05.293843 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.293825    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 20:25:05.293940 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.293864    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 20:25:05.296888 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.296859    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:25:05.314025 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.313960    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.315425 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.315406    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.315536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.315440    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.315536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.315453    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.315536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.315484    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.322015 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.321998    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.322064 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.322024    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-131.ec2.internal\": node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.348416 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.348368    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.395178 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.395135    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"]
Jul 28 20:25:05.395285 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.395250    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.397197 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.397181    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.397294 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.397214    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.397294 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.397225    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.398676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.398655    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.398777 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.398760    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.398847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.398801    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.399679 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399663    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.399753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399684    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.399753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399694    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.399753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399705    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.399753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399708    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.399753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.399720    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.401042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.401023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.401127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.401058    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 20:25:05.401970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.401953    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 20:25:05.402068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.401980    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 20:25:05.402068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.401995    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeHasSufficientPID"
Jul 28 20:25:05.420153 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.420127    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-131.ec2.internal\" not found" node="ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.424505 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.424487    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-131.ec2.internal\" not found" node="ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.445272 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.445236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.445272 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.445273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.445470 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.445298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29f9052eaaed67906c3e76b084a6751a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-131.ec2.internal\" (UID: \"29f9052eaaed67906c3e76b084a6751a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.448513 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.448492    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.545484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.545484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.545614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29f9052eaaed67906c3e76b084a6751a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-131.ec2.internal\" (UID: \"29f9052eaaed67906c3e76b084a6751a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.545614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29f9052eaaed67906c3e76b084a6751a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-131.ec2.internal\" (UID: \"29f9052eaaed67906c3e76b084a6751a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.545614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.545614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.545546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/acdf80f2ede405bcd6f69099df2716b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal\" (UID: \"acdf80f2ede405bcd6f69099df2716b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.548748 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.548725    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.649396 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.649286    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.728727 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.728700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.728815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:05.728786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:05.749694 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.749654    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.850254 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.850195    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:05.951431 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:05.951337    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:06.051809 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.051775    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-131.ec2.internal\" not found"
Jul 28 20:25:06.079002 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.078971    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 20:25:06.079478 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.079145    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 20:25:06.112300 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.112272    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:25:06.114798 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.114778    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:25:06.120031 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.120019    2576 apiserver.go:52] "Watching apiserver"
Jul 28 20:25:06.128401 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.128367    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 20:25:06.130677 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.129086    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6746s","kube-system/konnectivity-agent-2hz8x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp","openshift-cluster-node-tuning-operator/tuned-xqn8d","openshift-dns/node-resolver-mpvlz","openshift-multus/multus-additional-cni-plugins-9lb2j","openshift-network-operator/iptables-alerter-cdw6l","openshift-image-registry/node-ca-mbvjh","openshift-multus/multus-9j2qr","openshift-multus/network-metrics-daemon-kvbr4","openshift-network-diagnostics/network-check-target-fb2rf","openshift-ovn-kubernetes/ovnkube-node-wmcxq"]
Jul 28 20:25:06.131008 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.130987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.131108 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.131081    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:06.132308 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.132292    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.133683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.133663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.134533 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.134515    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-48fxh\""
Jul 28 20:25:06.134643 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.134628    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 20:25:06.134727 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.134703    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 20:25:06.134923 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.134906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.135679 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.135660    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.135941 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.135917    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z2g6f\""
Jul 28 20:25:06.136057 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136042    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.136158 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 20:25:06.136267 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.136902 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136880    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.136992 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136978    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.137068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.136996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qrj4q\""
Jul 28 20:25:06.137510 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.137494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.138273 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.138255    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.138404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.138380    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.138525 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.138510    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dkcjj\""
Jul 28 20:25:06.139289 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.139271    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.139544 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.139530    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 20:25:06.139603 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.139580    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.140025 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.140009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 20:25:06.140082 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.140068    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 20:25:06.140130 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.140113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vrljx\""
Jul 28 20:25:06.140209 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.140193    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.140489 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.140474    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.141475 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.141459    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2s5l5\""
Jul 28 20:25:06.141604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.141587    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.141744 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.141727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.141806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.141779    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 20:25:06.141806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.141784    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.142996 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.142981    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 20:25:06.143095 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143024    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:06.143145 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143105    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 20:25:06.143284 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143262    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sk7x9\""
Jul 28 20:25:06.143367 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143328    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.143367 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143343    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.143477 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.143425    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:06.143477 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.143348    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.144394 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.144378    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hcqjg\""
Jul 28 20:25:06.144508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.144447    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 20:25:06.144743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.144725    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:06.144812 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.144789    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:06.146210 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.146192    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.148403 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.148385    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 20:25:06.148598 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.148582    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 20:25:06.148679 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.148630    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 20:25:06.148744 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.148637    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 20:25:06.149525 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.149510    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vxsh9\""
Jul 28 20:25:06.149597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.149571    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 20:25:06.149704 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.149685    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 20:25:06.150826 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-multus\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.150925 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-tuned\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.150925 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3746f2e3-5279-4ff3-85af-b46d1718e6be-hosts-file\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.150925 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fc2c721-7268-47a3-90ea-465677616a67-agent-certs\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.150925 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-system-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150951    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cnibin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-hostroot\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.150994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4b4q\" (UniqueName: \"kubernetes.io/projected/22ce665f-5388-470c-aaae-8d0c7d5a9b09-kube-api-access-b4b4q\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpz8s\" (UniqueName: \"kubernetes.io/projected/f1438abd-a656-4896-9907-1281e01f9249-kube-api-access-hpz8s\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-run\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-tmp\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-k8s-cni-cncf-io\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-kubelet\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-kubelet-config\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-dbus\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3746f2e3-5279-4ff3-85af-b46d1718e6be-tmp-dir\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f5ll\" (UniqueName: \"kubernetes.io/projected/3746f2e3-5279-4ff3-85af-b46d1718e6be-kube-api-access-5f5ll\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-device-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cni-binary-copy\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-daemon-config\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-host\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.151509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-serviceca\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-etc-kubernetes\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-registration-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-etc-selinux\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v86lr\" (UniqueName: \"kubernetes.io/projected/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-kube-api-access-v86lr\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98e75661-90f3-461e-a38e-34e6e794d050-iptables-alerter-script\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-conf-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151714    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-multus-certs\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cnibin\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-modprobe-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysconfig\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-systemd\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-system-cni-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-netns\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.152157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-conf\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-sys\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151971    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-var-lib-kubelet\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.151986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7hwk\" (UniqueName: \"kubernetes.io/projected/98e75661-90f3-461e-a38e-34e6e794d050-kube-api-access-z7hwk\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzq67\" (UniqueName: \"kubernetes.io/projected/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-kube-api-access-gzq67\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-kube-api-access-w9692\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-os-release\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-socket-dir-parent\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-socket-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98e75661-90f3-461e-a38e-34e6e794d050-host-slash\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-bin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-kubernetes\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.153005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-lib-modules\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx9sb\" (UniqueName: \"kubernetes.io/projected/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-kube-api-access-tx9sb\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-os-release\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152482    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c721-7268-47a3-90ea-465677616a67-konnectivity-ca\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-host\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-sys-fs\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152736    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.152988    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"
Jul 28 20:25:06.155532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.153043    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal"]
Jul 28 20:25:06.156337 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.156307    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 20:25:06.158767 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.158736    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:25:06.160093 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.160072    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 20:25:06.160258 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.160238    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal"]
Jul 28 20:25:06.168600 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.168555    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 20:20:05 +0000 UTC" deadline="2028-04-01 15:35:52.810877571 +0000 UTC"
Jul 28 20:25:06.168600 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.168599    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14707h10m46.642282994s"
Jul 28 20:25:06.176185 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.176167    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k8dtt"
Jul 28 20:25:06.183689 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.183669    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k8dtt"
Jul 28 20:25:06.244723 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.244698    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 20:25:06.253559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-sys-fs\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2khth\" (UniqueName: \"kubernetes.io/projected/412aae3a-8411-4920-b85a-fad6f872de6c-kube-api-access-2khth\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-multus\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-tuned\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-netd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3746f2e3-5279-4ff3-85af-b46d1718e6be-hosts-file\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fc2c721-7268-47a3-90ea-465677616a67-agent-certs\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-multus\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-sys-fs\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-system-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cnibin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-hostroot\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4b4q\" (UniqueName: \"kubernetes.io/projected/22ce665f-5388-470c-aaae-8d0c7d5a9b09-kube-api-access-b4b4q\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-system-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3746f2e3-5279-4ff3-85af-b46d1718e6be-hosts-file\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.253793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpz8s\" (UniqueName: \"kubernetes.io/projected/f1438abd-a656-4896-9907-1281e01f9249-kube-api-access-hpz8s\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cnibin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-run\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-tmp\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253922    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253937    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-run\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-k8s-cni-cncf-io\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-k8s-cni-cncf-io\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.253999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-kubelet\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-kubelet-config\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-hostroot\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-dbus\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-kubelet\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-kubelet-config\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-kubelet\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3144fb9b-2e19-46b6-ad95-b25777f15fc2-dbus\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.254676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3746f2e3-5279-4ff3-85af-b46d1718e6be-tmp-dir\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254247    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-netns\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.254333    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-env-overrides\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.254516    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:06.754484044 +0000 UTC m=+1.939588778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f5ll\" (UniqueName: \"kubernetes.io/projected/3746f2e3-5279-4ff3-85af-b46d1718e6be-kube-api-access-5f5ll\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-device-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-systemd-units\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3746f2e3-5279-4ff3-85af-b46d1718e6be-tmp-dir\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-systemd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-script-lib\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cni-binary-copy\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-daemon-config\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-host\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.255563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-serviceca\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-etc-kubernetes\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-registration-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-etc-selinux\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-device-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.254982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-config\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-host\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v86lr\" (UniqueName: \"kubernetes.io/projected/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-kube-api-access-v86lr\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98e75661-90f3-461e-a38e-34e6e794d050-iptables-alerter-script\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-conf-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-multus-certs\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-slash\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cnibin\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-modprobe-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysconfig\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-systemd\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.256456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-system-cni-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-netns\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-etc-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-etc-selinux\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-system-cni-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-netns\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-conf-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-etc-kubernetes\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-cnibin\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-conf\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-run-multus-certs\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-sys\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-var-lib-kubelet\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.257298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-conf\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.255992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hwk\" (UniqueName: \"kubernetes.io/projected/98e75661-90f3-461e-a38e-34e6e794d050-kube-api-access-z7hwk\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-registration-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzq67\" (UniqueName: \"kubernetes.io/projected/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-kube-api-access-gzq67\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-serviceca\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-kube-api-access-w9692\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysctl-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-sysconfig\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-os-release\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-socket-dir-parent\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-os-release\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-socket-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-sys\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/98e75661-90f3-461e-a38e-34e6e794d050-iptables-alerter-script\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-var-lib-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.258036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-node-log\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-bin\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-systemd\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98e75661-90f3-461e-a38e-34e6e794d050-host-slash\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-bin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-ovn\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-modprobe-d\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-kubernetes\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-socket-dir-parent\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-lib-modules\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98e75661-90f3-461e-a38e-34e6e794d050-host-slash\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-kubernetes\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx9sb\" (UniqueName: \"kubernetes.io/projected/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-kube-api-access-tx9sb\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.258629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1438abd-a656-4896-9907-1281e01f9249-socket-dir\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-cni-dir\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.256977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-log-socket\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412aae3a-8411-4920-b85a-fad6f872de6c-ovn-node-metrics-cert\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-os-release\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c721-7268-47a3-90ea-465677616a67-konnectivity-ca\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-host\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-os-release\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.257188    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.257257    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:06.757240178 +0000 UTC m=+1.942344899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-host\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-lib-modules\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-var-lib-kubelet\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22ce665f-5388-470c-aaae-8d0c7d5a9b09-host-var-lib-cni-bin\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-multus-daemon-config\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce665f-5388-470c-aaae-8d0c7d5a9b09-cni-binary-copy\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.259349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9fc2c721-7268-47a3-90ea-465677616a67-konnectivity-ca\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.259968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.257968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.259968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.258164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-etc-tuned\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.259968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.258230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-tmp\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.259968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.258230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9fc2c721-7268-47a3-90ea-465677616a67-agent-certs\") pod \"konnectivity-agent-2hz8x\" (UID: \"9fc2c721-7268-47a3-90ea-465677616a67\") " pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.267414 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.267382    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:06.267414 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.267412    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:06.267596 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.267426    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:06.267689 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.267672    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:06.767620895 +0000 UTC m=+1.952725595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:06.268211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.268085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f5ll\" (UniqueName: \"kubernetes.io/projected/3746f2e3-5279-4ff3-85af-b46d1718e6be-kube-api-access-5f5ll\") pod \"node-resolver-mpvlz\" (UID: \"3746f2e3-5279-4ff3-85af-b46d1718e6be\") " pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.268211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.268133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpz8s\" (UniqueName: \"kubernetes.io/projected/f1438abd-a656-4896-9907-1281e01f9249-kube-api-access-hpz8s\") pod \"aws-ebs-csi-driver-node-j8bsp\" (UID: \"f1438abd-a656-4896-9907-1281e01f9249\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.268376 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.268220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4b4q\" (UniqueName: \"kubernetes.io/projected/22ce665f-5388-470c-aaae-8d0c7d5a9b09-kube-api-access-b4b4q\") pod \"multus-9j2qr\" (UID: \"22ce665f-5388-470c-aaae-8d0c7d5a9b09\") " pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.269422 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.269390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v86lr\" (UniqueName: \"kubernetes.io/projected/ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29-kube-api-access-v86lr\") pod \"multus-additional-cni-plugins-9lb2j\" (UID: \"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29\") " pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.269915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.269887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hwk\" (UniqueName: \"kubernetes.io/projected/98e75661-90f3-461e-a38e-34e6e794d050-kube-api-access-z7hwk\") pod \"iptables-alerter-cdw6l\" (UID: \"98e75661-90f3-461e-a38e-34e6e794d050\") " pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.270530 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.270506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/27f27e8a-e6e3-40c1-afda-6a333b30e6e2-kube-api-access-w9692\") pod \"tuned-xqn8d\" (UID: \"27f27e8a-e6e3-40c1-afda-6a333b30e6e2\") " pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.271233 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.271215    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzq67\" (UniqueName: \"kubernetes.io/projected/b23d94ba-ba07-4fa3-9fa7-94ce25677eeb-kube-api-access-gzq67\") pod \"node-ca-mbvjh\" (UID: \"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb\") " pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.271503 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.271482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx9sb\" (UniqueName: \"kubernetes.io/projected/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-kube-api-access-tx9sb\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.286034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.286010    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9j2qr"
Jul 28 20:25:06.347341 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.347300    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdf80f2ede405bcd6f69099df2716b1.slice/crio-acfb48163388bda3e37c88114a7746434dea1fd56a998287ff8482d2e6b335f7 WatchSource:0}: Error finding container acfb48163388bda3e37c88114a7746434dea1fd56a998287ff8482d2e6b335f7: Status 404 returned error can't find the container with id acfb48163388bda3e37c88114a7746434dea1fd56a998287ff8482d2e6b335f7
Jul 28 20:25:06.356518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.356499    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:25:06.357521 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-netd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357587 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-netd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357732 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-kubelet\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-netns\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-env-overrides\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-run-netns\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-systemd-units\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-kubelet\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-systemd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.357858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-script-lib\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-config\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.357974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-slash\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358005    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-etc-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-var-lib-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-node-log\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-bin\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-ovn\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-systemd\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-cni-bin\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-log-socket\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-host-slash\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-node-log\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412aae3a-8411-4920-b85a-fad6f872de6c-ovn-node-metrics-cert\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-env-overrides\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-var-lib-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2khth\" (UniqueName: \"kubernetes.io/projected/412aae3a-8411-4920-b85a-fad6f872de6c-kube-api-access-2khth\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-etc-openvswitch\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-run-ovn\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-systemd-units\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.358675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-script-lib\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.359134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/412aae3a-8411-4920-b85a-fad6f872de6c-log-socket\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.359134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.358866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/412aae3a-8411-4920-b85a-fad6f872de6c-ovnkube-config\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.360899 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.360876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/412aae3a-8411-4920-b85a-fad6f872de6c-ovn-node-metrics-cert\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.365871 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.365851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f9052eaaed67906c3e76b084a6751a.slice/crio-4fb121b652a8c387ff19a3557d9dfcf84c85fd3b91aeefb327d8640d70412d89 WatchSource:0}: Error finding container 4fb121b652a8c387ff19a3557d9dfcf84c85fd3b91aeefb327d8640d70412d89: Status 404 returned error can't find the container with id 4fb121b652a8c387ff19a3557d9dfcf84c85fd3b91aeefb327d8640d70412d89
Jul 28 20:25:06.367540 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.367523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khth\" (UniqueName: \"kubernetes.io/projected/412aae3a-8411-4920-b85a-fad6f872de6c-kube-api-access-2khth\") pod \"ovnkube-node-wmcxq\" (UID: \"412aae3a-8411-4920-b85a-fad6f872de6c\") " pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.462796 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.462708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:06.471024 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.470995    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc2c721_7268_47a3_90ea_465677616a67.slice/crio-cf34b81a52b2a8f65df45c7b5094fd7bcb44c781f2245dd0020cb1e3220912c8 WatchSource:0}: Error finding container cf34b81a52b2a8f65df45c7b5094fd7bcb44c781f2245dd0020cb1e3220912c8: Status 404 returned error can't find the container with id cf34b81a52b2a8f65df45c7b5094fd7bcb44c781f2245dd0020cb1e3220912c8
Jul 28 20:25:06.486764 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.486742    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp"
Jul 28 20:25:06.494355 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.494319    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1438abd_a656_4896_9907_1281e01f9249.slice/crio-9f64222b5b0406aee37694bf6fc5d45ccc1bda165da8852fbbd5c8a95af982a9 WatchSource:0}: Error finding container 9f64222b5b0406aee37694bf6fc5d45ccc1bda165da8852fbbd5c8a95af982a9: Status 404 returned error can't find the container with id 9f64222b5b0406aee37694bf6fc5d45ccc1bda165da8852fbbd5c8a95af982a9
Jul 28 20:25:06.504615 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.504595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xqn8d"
Jul 28 20:25:06.508374 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.508342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpvlz"
Jul 28 20:25:06.512816 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.512789    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27f27e8a_e6e3_40c1_afda_6a333b30e6e2.slice/crio-24f17c532eb443270d26900b3da41db9994e32c6c1a7a8da4f3a3932ff141634 WatchSource:0}: Error finding container 24f17c532eb443270d26900b3da41db9994e32c6c1a7a8da4f3a3932ff141634: Status 404 returned error can't find the container with id 24f17c532eb443270d26900b3da41db9994e32c6c1a7a8da4f3a3932ff141634
Jul 28 20:25:06.516685 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.516661    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3746f2e3_5279_4ff3_85af_b46d1718e6be.slice/crio-ea93edb89cb12a34c8740fbbde2a61becf0f2bd46b44b36d290ff2f9aa75a447 WatchSource:0}: Error finding container ea93edb89cb12a34c8740fbbde2a61becf0f2bd46b44b36d290ff2f9aa75a447: Status 404 returned error can't find the container with id ea93edb89cb12a34c8740fbbde2a61becf0f2bd46b44b36d290ff2f9aa75a447
Jul 28 20:25:06.521921 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.521900    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lb2j"
Jul 28 20:25:06.529776 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.529749    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6b1fe1_dce1_4aa9_bb2a_8c4bce67ed29.slice/crio-f52f6e640189bf717227c3537ce0a4513bce1f1d923826edeafcaa7906727e33 WatchSource:0}: Error finding container f52f6e640189bf717227c3537ce0a4513bce1f1d923826edeafcaa7906727e33: Status 404 returned error can't find the container with id f52f6e640189bf717227c3537ce0a4513bce1f1d923826edeafcaa7906727e33
Jul 28 20:25:06.539528 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.539507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cdw6l"
Jul 28 20:25:06.549301 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.549279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e75661_90f3_461e_a38e_34e6e794d050.slice/crio-62d268977b3dbd01c7f5cde86630bd94c3203aac6111ca681040554306071cfc WatchSource:0}: Error finding container 62d268977b3dbd01c7f5cde86630bd94c3203aac6111ca681040554306071cfc: Status 404 returned error can't find the container with id 62d268977b3dbd01c7f5cde86630bd94c3203aac6111ca681040554306071cfc
Jul 28 20:25:06.567378 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.567346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbvjh"
Jul 28 20:25:06.574882 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.574857    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23d94ba_ba07_4fa3_9fa7_94ce25677eeb.slice/crio-9bf733061cfd4acfb5c341a66ba09b60eac96432bfd43c628d9365e14b883c09 WatchSource:0}: Error finding container 9bf733061cfd4acfb5c341a66ba09b60eac96432bfd43c628d9365e14b883c09: Status 404 returned error can't find the container with id 9bf733061cfd4acfb5c341a66ba09b60eac96432bfd43c628d9365e14b883c09
Jul 28 20:25:06.603045 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.603011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:06.611355 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.611330    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412aae3a_8411_4920_b85a_fad6f872de6c.slice/crio-b7200c6f38ca590e66f07aeab0d94ee22572c9c92ed942bb5b8feb23b8a5009d WatchSource:0}: Error finding container b7200c6f38ca590e66f07aeab0d94ee22572c9c92ed942bb5b8feb23b8a5009d: Status 404 returned error can't find the container with id b7200c6f38ca590e66f07aeab0d94ee22572c9c92ed942bb5b8feb23b8a5009d
Jul 28 20:25:06.702600 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:06.702556    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ce665f_5388_470c_aaae_8d0c7d5a9b09.slice/crio-abebf12aeefd91ffe70b1fbeb8471f7f1c2a89130d52521e03608444590e69a4 WatchSource:0}: Error finding container abebf12aeefd91ffe70b1fbeb8471f7f1c2a89130d52521e03608444590e69a4: Status 404 returned error can't find the container with id abebf12aeefd91ffe70b1fbeb8471f7f1c2a89130d52521e03608444590e69a4
Jul 28 20:25:06.761444 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.761402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:06.761624 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.761491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:06.761624 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.761562    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:06.761624 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.761616    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:06.761760 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.761633    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:07.761616786 +0000 UTC m=+2.946721493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:06.761760 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.761672    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:07.761654767 +0000 UTC m=+2.946759471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:06.862375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:06.862321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:06.862581 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.862500    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:06.862581 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.862518    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:06.862581 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.862528    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:06.862773 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:06.862588    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:07.862569874 +0000 UTC m=+3.047674560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:07.186132 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.185622    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:20:06 +0000 UTC" deadline="2028-03-01 03:20:52.402051807 +0000 UTC"
Jul 28 20:25:07.186132 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.185666    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13950h55m45.216390927s"
Jul 28 20:25:07.332344 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.332279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9j2qr" event={"ID":"22ce665f-5388-470c-aaae-8d0c7d5a9b09","Type":"ContainerStarted","Data":"abebf12aeefd91ffe70b1fbeb8471f7f1c2a89130d52521e03608444590e69a4"}
Jul 28 20:25:07.348774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.348733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"b7200c6f38ca590e66f07aeab0d94ee22572c9c92ed942bb5b8feb23b8a5009d"}
Jul 28 20:25:07.359597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.359501    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbvjh" event={"ID":"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb","Type":"ContainerStarted","Data":"9bf733061cfd4acfb5c341a66ba09b60eac96432bfd43c628d9365e14b883c09"}
Jul 28 20:25:07.383706 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.383659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cdw6l" event={"ID":"98e75661-90f3-461e-a38e-34e6e794d050","Type":"ContainerStarted","Data":"62d268977b3dbd01c7f5cde86630bd94c3203aac6111ca681040554306071cfc"}
Jul 28 20:25:07.405067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.405025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerStarted","Data":"f52f6e640189bf717227c3537ce0a4513bce1f1d923826edeafcaa7906727e33"}
Jul 28 20:25:07.410017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.409982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xqn8d" event={"ID":"27f27e8a-e6e3-40c1-afda-6a333b30e6e2","Type":"ContainerStarted","Data":"24f17c532eb443270d26900b3da41db9994e32c6c1a7a8da4f3a3932ff141634"}
Jul 28 20:25:07.415644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.415607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp" event={"ID":"f1438abd-a656-4896-9907-1281e01f9249","Type":"ContainerStarted","Data":"9f64222b5b0406aee37694bf6fc5d45ccc1bda165da8852fbbd5c8a95af982a9"}
Jul 28 20:25:07.436342 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.436252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2hz8x" event={"ID":"9fc2c721-7268-47a3-90ea-465677616a67","Type":"ContainerStarted","Data":"cf34b81a52b2a8f65df45c7b5094fd7bcb44c781f2245dd0020cb1e3220912c8"}
Jul 28 20:25:07.467580 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.467349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpvlz" event={"ID":"3746f2e3-5279-4ff3-85af-b46d1718e6be","Type":"ContainerStarted","Data":"ea93edb89cb12a34c8740fbbde2a61becf0f2bd46b44b36d290ff2f9aa75a447"}
Jul 28 20:25:07.477218 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.476912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal" event={"ID":"29f9052eaaed67906c3e76b084a6751a","Type":"ContainerStarted","Data":"4fb121b652a8c387ff19a3557d9dfcf84c85fd3b91aeefb327d8640d70412d89"}
Jul 28 20:25:07.496738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.496691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal" event={"ID":"acdf80f2ede405bcd6f69099df2716b1","Type":"ContainerStarted","Data":"acfb48163388bda3e37c88114a7746434dea1fd56a998287ff8482d2e6b335f7"}
Jul 28 20:25:07.501953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.501927    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.772222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.772293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.772471    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.772536    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:09.772515911 +0000 UTC m=+4.957620600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.773065    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:07.773170 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.773122    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:09.773104791 +0000 UTC m=+4.958209478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:07.873689 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:07.873653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:07.873965 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.873886    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:07.873965 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.873908    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:07.873965 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.873921    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:07.874193 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:07.873982    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:09.873962583 +0000 UTC m=+5.059067288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:08.186463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:08.186314    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 20:20:06 +0000 UTC" deadline="2028-04-01 21:38:27.609358218 +0000 UTC"
Jul 28 20:25:08.186463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:08.186394    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14713h13m19.422968529s"
Jul 28 20:25:08.294718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:08.294680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:08.294908 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:08.294835    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:08.295304 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:08.295279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:08.295406 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:08.295390    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:08.295497 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:08.295480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:08.295580 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:08.295559    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:09.790547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:09.790501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:09.791112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:09.790593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:09.791112 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.790756    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:09.791112 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.790825    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:13.790805042 +0000 UTC m=+8.975909734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:09.791284 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.791237    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:09.791326 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.791288    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:13.791272385 +0000 UTC m=+8.976377075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:09.891674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:09.891516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:09.891884 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.891752    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:09.891884 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.891786    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:09.891884 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.891802    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:09.891884 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:09.891868    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:13.891845343 +0000 UTC m=+9.076950034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:10.294953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:10.294918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:10.295159 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:10.294965    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:10.295159 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:10.294918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:10.295159 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:10.295060    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:10.295366 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:10.295187    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:10.295366 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:10.295293    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:12.294882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:12.295048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:12.295511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:12.295628    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:12.295685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:12.295808 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:12.295756    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:13.829868 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:13.829827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:13.830360 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:13.829908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:13.830360 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.830036    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:13.830360 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.830109    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:21.830088385 +0000 UTC m=+17.015193091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:13.830360 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.830038    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:13.830360 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.830204    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:21.830185752 +0000 UTC m=+17.015290438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:13.931352 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:13.930622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:13.931352 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.930862    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:13.931352 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.930886    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:13.931352 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.930902    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:13.931352 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:13.930969    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:21.930948006 +0000 UTC m=+17.116052692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:14.294333 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:14.294243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:14.294555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:14.294393    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:14.294555 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:14.294384    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:14.294555 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:14.294539    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:14.294735 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:14.294607    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:14.294735 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:14.294681    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:16.294841 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:16.294803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:16.295280 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:16.294968    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:16.295280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:16.295057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:16.295280 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:16.295177    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:16.295280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:16.295228    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:16.295478 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:16.295295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:18.294380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:18.294327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:18.294380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:18.294378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:18.294883 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:18.294381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:18.294883 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:18.294474    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:18.294883 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:18.294544    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:18.294883 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:18.294662    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:20.294198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:20.294151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:20.294198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:20.294197    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:20.294713 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:20.294174    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:20.294713 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:20.294282    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:20.294713 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:20.294397    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:20.294713 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:20.294466    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:21.890822 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:21.890786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:21.891300 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:21.890842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:21.891300 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.890953    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:21.891300 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.890961    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:21.891300 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.891014    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:25:37.890995867 +0000 UTC m=+33.076100553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:21.891300 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.891029    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:37.891022762 +0000 UTC m=+33.076127446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:21.991412 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:21.991375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:21.991613 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.991592    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:21.991670 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.991625    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:21.991670 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.991640    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:21.991744 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:21.991706    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:25:37.991686366 +0000 UTC m=+33.176791062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:22.294106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:22.294063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:22.294387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:22.294063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:22.294387 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:22.294318    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:22.294387 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:22.294214    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:22.294387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:22.294063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:22.294604 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:22.294416    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:24.294492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:24.294458    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:24.294492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:24.294480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:24.295002 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:24.294598    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:24.295002 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:24.294599    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:24.295002 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:24.294670    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:24.295002 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:24.294780    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:26.295345 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:26.295085    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:26.295780 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:26.295113    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:26.295780 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:26.295141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:26.295780 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:26.295508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:26.295780 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:26.295448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:26.295780 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:26.295564    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:28.295111 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.294920    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:28.295955 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.294960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:28.295955 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:28.295203    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:28.295955 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:28.295293    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:28.295955 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.294921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:28.295955 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:28.295407    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:28.577558 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"8ed6008eae17e0d6d3465a18d30a102182f3b8dc173c9816456c0d82fae060c8"}
Jul 28 20:25:28.577749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"e56fd9f8ca75d78ebdd5f875b28f444ca0331964f19e33e8644927d4de8d6aad"}
Jul 28 20:25:28.577749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"032ed0616726d787eb059cce27905ec685f01ad3e7e92851f5faddc37f862c3b"}
Jul 28 20:25:28.577749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577705    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"0efd38646a09dc638fdb2615dca5d2a16abd91e22ef0d91502942c9515b38421"}
Jul 28 20:25:28.577895 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"6c368dcde78dc711f44d1bd8572f77c029e10b6cf5a7fbbc12c751674b736b2f"}
Jul 28 20:25:28.577895 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.577797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"26d8495c219f93738556a2dc96fc727f47b447a54753cbe59b72342354fb3031"}
Jul 28 20:25:28.579133 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.579096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbvjh" event={"ID":"b23d94ba-ba07-4fa3-9fa7-94ce25677eeb","Type":"ContainerStarted","Data":"a4b3e4c6a78954766eab9caf7e262b11dfb017d6d0bb12dc7072afc749ad68af"}
Jul 28 20:25:28.580851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.580821    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="1f4a77ed40cc9df260c2af8ca400275ca661a9db12a4443935d57a1fb1c05564" exitCode=0
Jul 28 20:25:28.580952 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.580904    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"1f4a77ed40cc9df260c2af8ca400275ca661a9db12a4443935d57a1fb1c05564"}
Jul 28 20:25:28.582554 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.582530    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xqn8d" event={"ID":"27f27e8a-e6e3-40c1-afda-6a333b30e6e2","Type":"ContainerStarted","Data":"24f911b2815cba56f325c26a6a0260cea1af4a006404a08213be7f0fcd6c4685"}
Jul 28 20:25:28.584195 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.584172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp" event={"ID":"f1438abd-a656-4896-9907-1281e01f9249","Type":"ContainerStarted","Data":"13b06323fc6ac31056934f85b57b9198672d044d826504b910d57fd176894dbe"}
Jul 28 20:25:28.585778 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.585758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2hz8x" event={"ID":"9fc2c721-7268-47a3-90ea-465677616a67","Type":"ContainerStarted","Data":"e12efaf479a9d0582b55018da6f2d21eec26917131d89d808ca3ed4514d8f1ef"}
Jul 28 20:25:28.587463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.587395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpvlz" event={"ID":"3746f2e3-5279-4ff3-85af-b46d1718e6be","Type":"ContainerStarted","Data":"60905a25a0ba5383cb78e8984a071b28dd9297a5b7995ccfade64687a06181db"}
Jul 28 20:25:28.588866 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.588845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal" event={"ID":"29f9052eaaed67906c3e76b084a6751a","Type":"ContainerStarted","Data":"ce8bd914f45a2d74d574113aa1297adebdb53ee8db9a47001027b843c5363e70"}
Jul 28 20:25:28.590459 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.590425    2576 generic.go:356] "Generic (PLEG): container finished" podID="acdf80f2ede405bcd6f69099df2716b1" containerID="03fad44a6024e92ba5fb5e44000d816926e10929fa06397d2e179cb4bc3a1d40" exitCode=0
Jul 28 20:25:28.590554 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.590475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal" event={"ID":"acdf80f2ede405bcd6f69099df2716b1","Type":"ContainerDied","Data":"03fad44a6024e92ba5fb5e44000d816926e10929fa06397d2e179cb4bc3a1d40"}
Jul 28 20:25:28.591956 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.591936    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9j2qr" event={"ID":"22ce665f-5388-470c-aaae-8d0c7d5a9b09","Type":"ContainerStarted","Data":"7315f04711a58d49370b74014069b242a2d9a98964be2eb02af58077b6bb13c6"}
Jul 28 20:25:28.593705 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.593657    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mbvjh" podStartSLOduration=1.8437443629999999 podStartE2EDuration="22.59363421s" totalImagesPullingTime="20.749889847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.593066226 +0000 UTC m=+23.778170968" watchObservedRunningTime="2026-07-28 20:25:28.59363421 +0000 UTC m=+23.778738916"
Jul 28 20:25:28.608488 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.608442    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xqn8d" podStartSLOduration=1.770212968 podStartE2EDuration="22.608424786s" totalImagesPullingTime="20.838211818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.608425331 +0000 UTC m=+23.793530037" watchObservedRunningTime="2026-07-28 20:25:28.608424786 +0000 UTC m=+23.793529497"
Jul 28 20:25:28.621961 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.621917    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mpvlz" podStartSLOduration=1.792259358 podStartE2EDuration="22.621907154s" totalImagesPullingTime="20.829647796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.621530041 +0000 UTC m=+23.806634747" watchObservedRunningTime="2026-07-28 20:25:28.621907154 +0000 UTC m=+23.807011861"
Jul 28 20:25:28.648899 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.648862    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-131.ec2.internal" podStartSLOduration=22.648846739 podStartE2EDuration="22.648846739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.648464998 +0000 UTC m=+23.833569703" watchObservedRunningTime="2026-07-28 20:25:28.648846739 +0000 UTC m=+23.833951445"
Jul 28 20:25:28.686938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.686862    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9j2qr" podStartSLOduration=2.005808205 podStartE2EDuration="22.686847849s" totalImagesPullingTime="20.681039644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.686832604 +0000 UTC m=+23.871937306" watchObservedRunningTime="2026-07-28 20:25:28.686847849 +0000 UTC m=+23.871952555"
Jul 28 20:25:28.701127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.701086    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2hz8x" podStartSLOduration=1.870155732 podStartE2EDuration="22.701073548s" totalImagesPullingTime="20.830917816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:28.700769332 +0000 UTC m=+23.885874039" watchObservedRunningTime="2026-07-28 20:25:28.701073548 +0000 UTC m=+23.886178284"
Jul 28 20:25:28.747463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.747425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:28.809085 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.808899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:28.909276 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:28.909240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:29.049304 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.049279    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 20:25:29.226158 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.225944    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T20:25:29.049314048Z","UUID":"71494da7-01f9-4be5-a2e0-16984f4015a8","Handler":null,"Name":"","Endpoint":""}
Jul 28 20:25:29.227610 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.227589    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 20:25:29.227738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.227631    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 20:25:29.596836 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.596798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cdw6l" event={"ID":"98e75661-90f3-461e-a38e-34e6e794d050","Type":"ContainerStarted","Data":"23d6bb70747fdff27972fe8edd6df9570e507787c56d26c71b51491d6d792889"}
Jul 28 20:25:29.599449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.599416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp" event={"ID":"f1438abd-a656-4896-9907-1281e01f9249","Type":"ContainerStarted","Data":"156838a7c11a4315c0c8d5104f46313489fd93dc0cab34ab92cf34181b165c90"}
Jul 28 20:25:29.601938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.601913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal" event={"ID":"acdf80f2ede405bcd6f69099df2716b1","Type":"ContainerStarted","Data":"328cb1e6bb89ced07ad2cb46afc1f596da1a4110da23a655aaf945a35abba911"}
Jul 28 20:25:29.613745 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.613696    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cdw6l" podStartSLOduration=2.861136194 podStartE2EDuration="23.613677472s" totalImagesPullingTime="20.752541278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:29.613412007 +0000 UTC m=+24.798516715" watchObservedRunningTime="2026-07-28 20:25:29.613677472 +0000 UTC m=+24.798782181"
Jul 28 20:25:29.628397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.628330    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-131.ec2.internal" podStartSLOduration=23.628315429 podStartE2EDuration="23.628315429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:29.627818409 +0000 UTC m=+24.812923117" watchObservedRunningTime="2026-07-28 20:25:29.628315429 +0000 UTC m=+24.813420136"
Jul 28 20:25:29.835200 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.835121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:29.935426 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:29.935339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:30.294574 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.294546    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:30.294574 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.294571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:30.294769 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:30.294691    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:30.294918 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.294858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:30.295028 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:30.294976    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:30.295075 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:30.295053    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:30.609335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.609229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"be45aeeb3f6a6e613080a133166e92af21ee032bdbf1f1fc780f52c72ed787b3"}
Jul 28 20:25:30.611430 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.611399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp" event={"ID":"f1438abd-a656-4896-9907-1281e01f9249","Type":"ContainerStarted","Data":"b1ce6c7954f4b31b0e5a425dcb5e04e93a396f4b4cd2fa6a96aa3c3bab9c1b2c"}
Jul 28 20:25:30.635066 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.635012    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j8bsp" podStartSLOduration=1.7012660240000002 podStartE2EDuration="24.634996043s" totalImagesPullingTime="22.933730019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:30.634513356 +0000 UTC m=+25.819618063" watchObservedRunningTime="2026-07-28 20:25:30.634996043 +0000 UTC m=+25.820100750"
Jul 28 20:25:30.768307 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.768271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:30.821667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:30.821612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:31.601560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:31.601524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:31.887839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:31.887758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.092547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.092513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.188517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.188424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.205413 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.205387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:32.205927 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.205901    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:32.294545 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.294515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:32.294729 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.294630    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:32.294729 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:32.294636    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:32.294729 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.294682    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:32.294852 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:32.294787    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:32.294886 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:32.294873    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:32.348499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.348462    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.448451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.448359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.616525 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.616500    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:32.616958 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.616941    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2hz8x"
Jul 28 20:25:32.645274 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.645242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.754322 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.754148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.856798 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.856769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:32.962921 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:32.962842    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpvlz_3746f2e3-5279-4ff3-85af-b46d1718e6be/dns-node-resolver/0.log"
Jul 28 20:25:33.621554 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.621514    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" event={"ID":"412aae3a-8411-4920-b85a-fad6f872de6c","Type":"ContainerStarted","Data":"455389037db370a866278244a6666675d911a89df82f0c6f65ec8346db319bcc"}
Jul 28 20:25:33.621744 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.621728    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:33.623255 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.623229    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="db10b4aca79b3ddc89f1fdb90ae786f365a43b62cf6d3f7c27aa282a178ccd14" exitCode=0
Jul 28 20:25:33.623400 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.623304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"db10b4aca79b3ddc89f1fdb90ae786f365a43b62cf6d3f7c27aa282a178ccd14"}
Jul 28 20:25:33.637576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.637555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:33.648860 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.648817    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq" podStartSLOduration=6.767034504 podStartE2EDuration="27.64880463s" totalImagesPullingTime="20.881770126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:33.648312585 +0000 UTC m=+28.833417292" watchObservedRunningTime="2026-07-28 20:25:33.64880463 +0000 UTC m=+28.833909332"
Jul 28 20:25:33.964387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:33.964283    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mbvjh_b23d94ba-ba07-4fa3-9fa7-94ce25677eeb/node-ca/0.log"
Jul 28 20:25:34.294291 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.294263    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:34.294534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.294344    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:34.294534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.294377    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:34.294534 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.294489    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:34.294697 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.294537    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:34.294697 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.294613    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:34.324968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.324925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:34.414961 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.414925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:34.625947 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.625922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:34.626263 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.626242    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:34.640914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.640890    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:25:34.730016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.729878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:34.832067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.831965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:34.906109 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.906077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvbr4"]
Jul 28 20:25:34.906269 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.906202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:34.906344 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.906321    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:34.909284 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.909254    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6746s"]
Jul 28 20:25:34.909430 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.909343    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:34.909501 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.909443    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:34.909962 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.909943    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb2rf"]
Jul 28 20:25:34.910054 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:34.910032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:34.910177 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:34.910159    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:35.147238 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:35.147154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:35.630352 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:35.630312    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="e38dbaf04cf002e98d13fcc6190b19b0f0a993819fc0b7267f1184b51efbc70e" exitCode=0
Jul 28 20:25:35.630535 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:35.630404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"e38dbaf04cf002e98d13fcc6190b19b0f0a993819fc0b7267f1184b51efbc70e"}
Jul 28 20:25:36.005950 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.005915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.220068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.220034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.294122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.294045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:36.294288 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.294060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:36.294288 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:36.294166    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:36.294288 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:36.294223    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:36.318105 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.318069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.418401 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.418332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.539326 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.539297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.776837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.776805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:36.877743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:36.877710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:37.241973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.241945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:37.294933 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.294909    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:37.295079 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:37.295011    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:37.515505 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.515426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:37.628045 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.628013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:37.637659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.637630    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="5a845a82a7b05deea2a3cf0ebd13eb2badd2640d3b362ae361c0c90d0aa5e0f6" exitCode=0
Jul 28 20:25:37.637809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.637685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"5a845a82a7b05deea2a3cf0ebd13eb2badd2640d3b362ae361c0c90d0aa5e0f6"}
Jul 28 20:25:37.693520 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.693491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:37.909794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.909722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:37.909794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:37.909776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:37.909973 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:37.909836    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:37.909973 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:37.909850    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:37.909973 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:37.909895    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs podName:a10a7ae8-3062-40b0-8c82-aa7e31c93b7f nodeName:}" failed. No retries permitted until 2026-07-28 20:26:09.90987867 +0000 UTC m=+65.094983355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs") pod "network-metrics-daemon-kvbr4" (UID: "a10a7ae8-3062-40b0-8c82-aa7e31c93b7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 20:25:37.909973 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:37.909923    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret podName:3144fb9b-2e19-46b6-ad95-b25777f15fc2 nodeName:}" failed. No retries permitted until 2026-07-28 20:26:09.909908056 +0000 UTC m=+65.095012755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret") pod "global-pull-secret-syncer-6746s" (UID: "3144fb9b-2e19-46b6-ad95-b25777f15fc2") : object "kube-system"/"original-pull-secret" not registered
Jul 28 20:25:38.010098 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.010063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:38.010260 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.010189    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 20:25:38.010260 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.010205    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 20:25:38.010260 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.010216    2576 projected.go:196] Error preparing data for projected volume kube-api-access-6jzmj for pod openshift-network-diagnostics/network-check-target-fb2rf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:38.010391 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.010269    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj podName:664da406-c952-48bf-acd5-577f1f39740d nodeName:}" failed. No retries permitted until 2026-07-28 20:26:10.01025459 +0000 UTC m=+65.195359276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jzmj" (UniqueName: "kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj") pod "network-check-target-fb2rf" (UID: "664da406-c952-48bf-acd5-577f1f39740d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 20:25:38.294346 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.294317    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:38.294703 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.294430    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:38.294703 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.294486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:38.294703 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:38.294565    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:38.299619 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.299590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.588157 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.588078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.679086 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.679045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:38.780243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:38.780198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.101348 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:39.101307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.294149 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:39.294112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:39.294329 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:39.294255    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:39.294481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:39.294435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.696202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:39.696162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:39.907148 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:39.907111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:40.005103 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.005057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:40.294961 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.294884    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:40.294961 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.294921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:40.295437 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:40.295031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:40.295437 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:40.295174    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:40.503494 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.503442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:40.698579 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.698497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:40.754306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:40.754265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:41.098959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:41.098926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:41.294690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:41.294649    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:41.294860 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:41.294779    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:41.298014 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:41.297923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:41.498083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:41.498047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:41.950848 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:41.950821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.261239 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.261206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.294223 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.294193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:42.294387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.294195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:42.294387 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:42.294295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:42.294465 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:42.294397    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:42.357061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.357032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.413498 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.413468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.652181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.652094    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="e3546daf5a91af754be290f7c17ec87bd69882da7ac0290e11b900f93d73a530" exitCode=0
Jul 28 20:25:42.652181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.652149    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"e3546daf5a91af754be290f7c17ec87bd69882da7ac0290e11b900f93d73a530"}
Jul 28 20:25:42.706504 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.706315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.806783 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.806746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:42.907738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:42.907658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.010073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.010038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.108405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.108355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.171325 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.171217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.295058 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.295025    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:43.295245 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:43.295144    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:43.316532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.316500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.417097 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.417053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.510394 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.510345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.657527 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.657497    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29" containerID="1f1c11cf3e791272caa65969903e84e63ef0aa3a6f1d3e34db09e31d9981a6fc" exitCode=0
Jul 28 20:25:43.657699 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.657565    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerDied","Data":"1f1c11cf3e791272caa65969903e84e63ef0aa3a6f1d3e34db09e31d9981a6fc"}
Jul 28 20:25:43.737432 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.737408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:43.936329 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:43.936231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:44.295007 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.294979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:44.295180 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.294979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:44.295180 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:44.295078    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:44.295180 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:44.295146    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:44.360647 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.360614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:44.418565 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.418526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:44.490857 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.490828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:44.663261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.663228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" event={"ID":"ef6b1fe1-dce1-4aa9-bb2a-8c4bce67ed29","Type":"ContainerStarted","Data":"1e0e89e2e4e101f02c86f2882564ab7fa246fe85265008f3de8911af2fbb98a2"}
Jul 28 20:25:44.686435 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.686387    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9lb2j" podStartSLOduration=7.557134835 podStartE2EDuration="38.686353819s" totalImagesPullingTime="31.129218984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:44.684516778 +0000 UTC m=+39.869621485" watchObservedRunningTime="2026-07-28 20:25:44.686353819 +0000 UTC m=+39.871458525"
Jul 28 20:25:44.898611 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.898531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:44.955291 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:44.955252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.092037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:45.091997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.293487 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:45.293455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:45.295175 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:45.295157    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:45.295265 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:45.295247    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:45.698422 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:45.698326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:46.294779 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:46.294745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:46.294995 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:46.294849    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:46.294995 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:46.294910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:46.295129 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:46.295031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:46.531402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:46.531351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:46.696824 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:46.696736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:46.897649 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:46.897616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:47.103866 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:47.103835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:47.294545 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:47.294514    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:47.294717 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:47.294618    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:48.294404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:48.294355    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:48.294782 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:48.294416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:48.294782 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:48.294487    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:48.294782 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:48.294606    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:48.570644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:48.570551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:48.670071 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:48.670044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:49.294763 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:49.294736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:49.295129 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:49.294827    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:50.294292 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:50.294259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:50.294480 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:50.294381    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:50.294480 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:50.294409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:50.294480 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:50.294459    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:51.294767 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:51.294735    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:51.295225 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:51.294833    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:52.294825 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:52.294785    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:52.295261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:52.294785    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:52.295261 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:52.294933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvbr4" podUID="a10a7ae8-3062-40b0-8c82-aa7e31c93b7f"
Jul 28 20:25:52.295261 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:52.295012    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6746s" podUID="3144fb9b-2e19-46b6-ad95-b25777f15fc2"
Jul 28 20:25:53.294874 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:53.294819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:53.295284 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:53.294946    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fb2rf" podUID="664da406-c952-48bf-acd5-577f1f39740d"
Jul 28 20:25:53.752406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:53.752357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.074543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.074454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.130483 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.130452    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-131.ec2.internal" event="NodeReady"
Jul 28 20:25:54.130627 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.130560    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 20:25:54.215069 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.215038    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"]
Jul 28 20:25:54.235937 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.235884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.238075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.238051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"]
Jul 28 20:25:54.238171 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.238139    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.243831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.243808    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 20:25:54.243946 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.243912    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 20:25:54.244343 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.244330    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 20:25:54.244923 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.244799    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-59v79\""
Jul 28 20:25:54.248228 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.248210    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vqjsk"]
Jul 28 20:25:54.257203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.257175    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 20:25:54.263806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.263787    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8w26d"]
Jul 28 20:25:54.263938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.263921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.267695 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.267672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 20:25:54.267787 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.267717    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 20:25:54.267787 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.267720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8kqlj\""
Jul 28 20:25:54.277325 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.277302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqjsk"]
Jul 28 20:25:54.277439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.277342    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8w26d"]
Jul 28 20:25:54.277439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.277422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.282325 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.282303    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 20:25:54.282441 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.282408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 20:25:54.282737 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.282704    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 20:25:54.283112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.282988    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ntb9d\""
Jul 28 20:25:54.294211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.294183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:25:54.294339 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.294322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:25:54.296968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.296939    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sn8zg\""
Jul 28 20:25:54.297260 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.296990    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:25:54.300089 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.300072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:25:54.331774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbl7f\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-kube-api-access-qbl7f\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.331774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-config-volume\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.331774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331765    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cc0a2c6-643f-43d4-a19c-75def068ffdf-ca-trust-extracted\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-trusted-ca\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zk6g\" (UniqueName: \"kubernetes.io/projected/234d4cc1-a8ce-40f7-80e7-423f8706623a-kube-api-access-5zk6g\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-tmp-dir\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-installation-pull-secrets\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-metrics-tls\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-tls\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.331973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-certificates\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.332235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.331990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-bound-sa-token\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.332235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.332007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234d4cc1-a8ce-40f7-80e7-423f8706623a-cert\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.332235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.332043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfnv\" (UniqueName: \"kubernetes.io/projected/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-kube-api-access-qcfnv\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.332235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.332085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-image-registry-private-configuration\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.365447 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.365415    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n5bbt"]
Jul 28 20:25:54.387063 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.387037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.392409 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.392241    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 20:25:54.392581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.392556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 20:25:54.395380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.395344    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 20:25:54.395641 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:54.395614    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-szcgk\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-szcgk\"" type="*v1.Secret"
Jul 28 20:25:54.395715 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:54.395630    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 28 20:25:54.410920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.410896    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5bbt"]
Jul 28 20:25:54.432349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cc0a2c6-643f-43d4-a19c-75def068ffdf-ca-trust-extracted\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.432349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-trusted-ca\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.432555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zk6g\" (UniqueName: \"kubernetes.io/projected/234d4cc1-a8ce-40f7-80e7-423f8706623a-kube-api-access-5zk6g\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.432555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ece9f1f-8051-4b3b-a516-60adc8b8df00-crio-socket\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.432555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-tmp-dir\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.432672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-installation-pull-secrets\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.432672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-metrics-tls\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.432672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-tls\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.432672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ece9f1f-8051-4b3b-a516-60adc8b8df00-data-volume\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.432672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-certificates\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-bound-sa-token\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234d4cc1-a8ce-40f7-80e7-423f8706623a-cert\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432707    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4cc0a2c6-643f-43d4-a19c-75def068ffdf-ca-trust-extracted\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfnv\" (UniqueName: \"kubernetes.io/projected/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-kube-api-access-qcfnv\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgdvz\" (UniqueName: \"kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-image-registry-private-configuration\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ece9f1f-8051-4b3b-a516-60adc8b8df00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbl7f\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-kube-api-access-qbl7f\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.432893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-config-volume\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.433042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.433016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-tmp-dir\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.433710 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.433326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-trusted-ca\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.433710 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.433438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-config-volume\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.433837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.433797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-certificates\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.435393 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.435341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.437463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.437429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-registry-tls\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.437553 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.437500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-metrics-tls\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.437595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.437561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-image-registry-private-configuration\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.437639 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.437605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234d4cc1-a8ce-40f7-80e7-423f8706623a-cert\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.437677 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.437647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4cc0a2c6-643f-43d4-a19c-75def068ffdf-installation-pull-secrets\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.461933 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.461905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zk6g\" (UniqueName: \"kubernetes.io/projected/234d4cc1-a8ce-40f7-80e7-423f8706623a-kube-api-access-5zk6g\") pod \"ingress-canary-8w26d\" (UID: \"234d4cc1-a8ce-40f7-80e7-423f8706623a\") " pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.462085 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.462067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbl7f\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-kube-api-access-qbl7f\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.467474 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.467453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cc0a2c6-643f-43d4-a19c-75def068ffdf-bound-sa-token\") pod \"image-registry-6cdd7ddc4b-ccfjl\" (UID: \"4cc0a2c6-643f-43d4-a19c-75def068ffdf\") " pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.470223 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.470202    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfnv\" (UniqueName: \"kubernetes.io/projected/52d5564d-7c12-46fb-bf0c-9d285bd5f7d9-kube-api-access-qcfnv\") pod \"dns-default-vqjsk\" (UID: \"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9\") " pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.534031 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.533997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ece9f1f-8051-4b3b-a516-60adc8b8df00-data-volume\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534031 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdvz\" (UniqueName: \"kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ece9f1f-8051-4b3b-a516-60adc8b8df00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ece9f1f-8051-4b3b-a516-60adc8b8df00-crio-socket\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534414 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1ece9f1f-8051-4b3b-a516-60adc8b8df00-crio-socket\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534480 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1ece9f1f-8051-4b3b-a516-60adc8b8df00-data-volume\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.534671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.534644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.536470 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.536452    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1ece9f1f-8051-4b3b-a516-60adc8b8df00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:54.547442 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.547420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:54.552974 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.552947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.572134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.572114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:25:54.586041 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.585984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8w26d"
Jul 28 20:25:54.687319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.686824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.782577 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.782542    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vqjsk"]
Jul 28 20:25:54.787106 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:54.787083    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d5564d_7c12_46fb_bf0c_9d285bd5f7d9.slice/crio-859f2ac7ae04e39597c07e6b632794ec3b00f8eaf8b0d9defd14c5116c9a5780 WatchSource:0}: Error finding container 859f2ac7ae04e39597c07e6b632794ec3b00f8eaf8b0d9defd14c5116c9a5780: Status 404 returned error can't find the container with id 859f2ac7ae04e39597c07e6b632794ec3b00f8eaf8b0d9defd14c5116c9a5780
Jul 28 20:25:54.791649 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.791538    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"]
Jul 28 20:25:54.793426 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:54.793403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc0a2c6_643f_43d4_a19c_75def068ffdf.slice/crio-599f8c3d022c2a554f6df4de65528dbd5fde8113c8472c758b9d762391dbbfea WatchSource:0}: Error finding container 599f8c3d022c2a554f6df4de65528dbd5fde8113c8472c758b9d762391dbbfea: Status 404 returned error can't find the container with id 599f8c3d022c2a554f6df4de65528dbd5fde8113c8472c758b9d762391dbbfea
Jul 28 20:25:54.810747 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.810723    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8w26d"]
Jul 28 20:25:54.813434 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:54.813403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234d4cc1_a8ce_40f7_80e7_423f8706623a.slice/crio-ff98b489244cb6020383bdb78f8ee8c5d4fa0005cb9e52cf623485a16d246524 WatchSource:0}: Error finding container ff98b489244cb6020383bdb78f8ee8c5d4fa0005cb9e52cf623485a16d246524: Status 404 returned error can't find the container with id ff98b489244cb6020383bdb78f8ee8c5d4fa0005cb9e52cf623485a16d246524
Jul 28 20:25:54.877219 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.877191    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:25:54.887096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.887074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.889534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.889515    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 20:25:54.889634 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.889569    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 20:25:54.890078 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890057    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 20:25:54.890166 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890084    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 20:25:54.890166 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 20:25:54.890319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890305    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 20:25:54.890376 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890318    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c5zqb\""
Jul 28 20:25:54.890614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.890582    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 20:25:54.894185 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.894079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:54.897902 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.897882    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:25:54.937266 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.937266 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.937519 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.937519 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.937519 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.937519 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.937456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdnq\" (UniqueName: \"kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:54.998342 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:54.998311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.038837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.038804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.038837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.038839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdnq\" (UniqueName: \"kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.039091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.038905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.039156 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.039080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.039156 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.039121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.039267 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.039170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.039962 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.039937    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.040070 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.040032    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.040114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.040073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.041441 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.041420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.041441 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.041431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.052335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.052310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdnq\" (UniqueName: \"kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq\") pod \"console-cf8bfd944-sgwhf\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") " pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.127468 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.127383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.197076 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.197030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:25:55.224016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.223981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.297982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.297949    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:25:55.301805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.301149    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9c5p9\""
Jul 28 20:25:55.301805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.301550    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:25:55.301805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.301789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:25:55.386474 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.386316    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:25:55.390425 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:55.390357    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58ded32_6d98_46f6_b4a0_156d70c69f2c.slice/crio-0e1aa02dab05e06f77e5eb49b75c468c0b528dddbb8a893a4a1541b4891f60c0 WatchSource:0}: Error finding container 0e1aa02dab05e06f77e5eb49b75c468c0b528dddbb8a893a4a1541b4891f60c0: Status 404 returned error can't find the container with id 0e1aa02dab05e06f77e5eb49b75c468c0b528dddbb8a893a4a1541b4891f60c0
Jul 28 20:25:55.431201 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.431163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.545496 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.545465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.553189 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:55.553155    2576 projected.go:291] Couldn't get configMap openshift-insights/openshift-service-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 28 20:25:55.553189 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:55.553181    2576 projected.go:196] Error preparing data for projected volume kube-api-access-dgdvz for pod openshift-insights/insights-runtime-extractor-n5bbt: failed to sync configmap cache: timed out waiting for the condition
Jul 28 20:25:55.553398 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:25:55.553257    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz podName:1ece9f1f-8051-4b3b-a516-60adc8b8df00 nodeName:}" failed. No retries permitted until 2026-07-28 20:25:56.053235211 +0000 UTC m=+51.238339898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dgdvz" (UniqueName: "kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz") pod "insights-runtime-extractor-n5bbt" (UID: "1ece9f1f-8051-4b3b-a516-60adc8b8df00") : failed to sync configmap cache: timed out waiting for the condition
Jul 28 20:25:55.645718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.645616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.660316 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.660285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-szcgk\""
Jul 28 20:25:55.677106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.677076    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 20:25:55.692822 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.692781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl" event={"ID":"4cc0a2c6-643f-43d4-a19c-75def068ffdf","Type":"ContainerStarted","Data":"f279abc3debe5a021969dc95f7386f419a67982a9a9b862eaebaabd7f13e76bb"}
Jul 28 20:25:55.692963 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.692841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl" event={"ID":"4cc0a2c6-643f-43d4-a19c-75def068ffdf","Type":"ContainerStarted","Data":"599f8c3d022c2a554f6df4de65528dbd5fde8113c8472c758b9d762391dbbfea"}
Jul 28 20:25:55.693022 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.693010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:25:55.694491 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.694459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf8bfd944-sgwhf" event={"ID":"c58ded32-6d98-46f6-b4a0-156d70c69f2c","Type":"ContainerStarted","Data":"0e1aa02dab05e06f77e5eb49b75c468c0b528dddbb8a893a4a1541b4891f60c0"}
Jul 28 20:25:55.696003 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.695952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8w26d" event={"ID":"234d4cc1-a8ce-40f7-80e7-423f8706623a","Type":"ContainerStarted","Data":"ff98b489244cb6020383bdb78f8ee8c5d4fa0005cb9e52cf623485a16d246524"}
Jul 28 20:25:55.698095 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.698042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqjsk" event={"ID":"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9","Type":"ContainerStarted","Data":"859f2ac7ae04e39597c07e6b632794ec3b00f8eaf8b0d9defd14c5116c9a5780"}
Jul 28 20:25:55.717915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:55.717833    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl" podStartSLOduration=2.7178150370000003 podStartE2EDuration="2.717815037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:25:55.717122237 +0000 UTC m=+50.902226945" watchObservedRunningTime="2026-07-28 20:25:55.717815037 +0000 UTC m=+50.902919746"
Jul 28 20:25:56.148304 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.148238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgdvz\" (UniqueName: \"kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:56.153818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.153677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgdvz\" (UniqueName: \"kubernetes.io/projected/1ece9f1f-8051-4b3b-a516-60adc8b8df00-kube-api-access-dgdvz\") pod \"insights-runtime-extractor-n5bbt\" (UID: \"1ece9f1f-8051-4b3b-a516-60adc8b8df00\") " pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:56.199087 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.199047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5bbt"
Jul 28 20:25:56.283456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.283419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:56.410404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.410304    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5bbt"]
Jul 28 20:25:56.458384 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.458339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:56.691570 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.691435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:56.792070 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:56.792024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.182447 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.181426    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:25:57.187658 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.187623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.197228 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.197205    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:25:57.197387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.197317    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.209093 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.209061    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 20:25:57.252878 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.252833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.261651 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261819 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261670    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzd88\" (UniqueName: \"kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261819 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261819 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.261971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.261957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzd88\" (UniqueName: \"kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363535    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.363811 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.363575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.365080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.364389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.365080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.364710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.365080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.364764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.365080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.365022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.366724 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.366695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.367336 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.367311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.376768 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.376719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzd88\" (UniqueName: \"kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88\") pod \"console-5b999d9bcd-dpbvp\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") " pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.479472 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.479392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.511317 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.511279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:25:57.560695 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.560660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.713313 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.713278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.814474 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.814380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.897473 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:57.897441    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ece9f1f_8051_4b3b_a516_60adc8b8df00.slice/crio-1a110cf4d92b403a6ddfc02668f4094ca0f14281e8375418b5cedf846b20ca08 WatchSource:0}: Error finding container 1a110cf4d92b403a6ddfc02668f4094ca0f14281e8375418b5cedf846b20ca08: Status 404 returned error can't find the container with id 1a110cf4d92b403a6ddfc02668f4094ca0f14281e8375418b5cedf846b20ca08
Jul 28 20:25:57.914462 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.914426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:57.980145 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:57.980119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:58.710182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:58.710146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5bbt" event={"ID":"1ece9f1f-8051-4b3b-a516-60adc8b8df00","Type":"ContainerStarted","Data":"1a110cf4d92b403a6ddfc02668f4094ca0f14281e8375418b5cedf846b20ca08"}
Jul 28 20:25:59.283963 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:59.283928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:59.342797 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:59.342763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:59.792871 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:25:59.792806    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:25:59.798963 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:25:59.798892    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e89c85b_dbfc_4805_978d_0be5351d6841.slice/crio-a78513ad5ca8f9b8b52cbbcdf57bc4b11233e5ccdfefb548dc4cd1e4e7446bb0 WatchSource:0}: Error finding container a78513ad5ca8f9b8b52cbbcdf57bc4b11233e5ccdfefb548dc4cd1e4e7446bb0: Status 404 returned error can't find the container with id a78513ad5ca8f9b8b52cbbcdf57bc4b11233e5ccdfefb548dc4cd1e4e7446bb0
Jul 28 20:26:00.718358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.718315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8w26d" event={"ID":"234d4cc1-a8ce-40f7-80e7-423f8706623a","Type":"ContainerStarted","Data":"fbd6caa8412aa92fa7a3187e59103cff22489feecb63f0286b08d07febf631e9"}
Jul 28 20:26:00.720572 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.720541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqjsk" event={"ID":"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9","Type":"ContainerStarted","Data":"c19f804d621ca24bf0c5066cabd2fa788f0b1c25a306b9632e2964d0dbb7b20c"}
Jul 28 20:26:00.720709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.720598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vqjsk" event={"ID":"52d5564d-7c12-46fb-bf0c-9d285bd5f7d9","Type":"ContainerStarted","Data":"7b1b06bc39606ed54fc9f5baca0ab222f41a3a1bec4f20d55086ab80b3bf555b"}
Jul 28 20:26:00.720709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.720632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:26:00.722233 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.722201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b999d9bcd-dpbvp" event={"ID":"7e89c85b-dbfc-4805-978d-0be5351d6841","Type":"ContainerStarted","Data":"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"}
Jul 28 20:26:00.722354 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.722248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b999d9bcd-dpbvp" event={"ID":"7e89c85b-dbfc-4805-978d-0be5351d6841","Type":"ContainerStarted","Data":"a78513ad5ca8f9b8b52cbbcdf57bc4b11233e5ccdfefb548dc4cd1e4e7446bb0"}
Jul 28 20:26:00.723964 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.723937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf8bfd944-sgwhf" event={"ID":"c58ded32-6d98-46f6-b4a0-156d70c69f2c","Type":"ContainerStarted","Data":"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"}
Jul 28 20:26:00.735065 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.735017    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8w26d" podStartSLOduration=1.905466241 podStartE2EDuration="6.734996975s" totalImagesPullingTime="4.829530734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:00.734650366 +0000 UTC m=+55.919755073" watchObservedRunningTime="2026-07-28 20:26:00.734996975 +0000 UTC m=+55.920101674"
Jul 28 20:26:00.755927 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.753433    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b999d9bcd-dpbvp" podStartSLOduration=3.753413711 podStartE2EDuration="3.753413711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:00.751456936 +0000 UTC m=+55.936561644" watchObservedRunningTime="2026-07-28 20:26:00.753413711 +0000 UTC m=+55.938518418"
Jul 28 20:26:00.770968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:00.770920    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cf8bfd944-sgwhf" podStartSLOduration=2.506103112 podStartE2EDuration="6.770903906s" totalImagesPullingTime="4.264800794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:00.770617634 +0000 UTC m=+55.955722342" watchObservedRunningTime="2026-07-28 20:26:00.770903906 +0000 UTC m=+55.956008612"
Jul 28 20:26:04.739868 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:04.739831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5bbt" event={"ID":"1ece9f1f-8051-4b3b-a516-60adc8b8df00","Type":"ContainerStarted","Data":"f4d9a358c5ed5ed3171db41b759b79db191e225c520d4ae86ee2a0438c12cd07"}
Jul 28 20:26:04.972201 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:04.972164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:05.067208 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:05.067114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:05.198391 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:05.198343    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:05.198585 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:05.198460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:05.199909 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:05.199869    2576 patch_prober.go:28] interesting pod/console-cf8bfd944-sgwhf container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused" start-of-body=
Jul 28 20:26:05.200042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:05.199925    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-cf8bfd944-sgwhf" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused"
Jul 28 20:26:06.649847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:06.649818    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wmcxq"
Jul 28 20:26:06.679936 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:06.679870    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vqjsk" podStartSLOduration=7.821221442 podStartE2EDuration="12.679846132s" totalImagesPullingTime="4.85862469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:00.786419491 +0000 UTC m=+55.971524198" watchObservedRunningTime="2026-07-28 20:26:06.679846132 +0000 UTC m=+61.864950838"
Jul 28 20:26:06.806384 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:06.806334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:06.865643 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:06.865603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:07.512210 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.512169    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:07.512404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.512246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:07.513688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.513651    2576 patch_prober.go:28] interesting pod/console-5b999d9bcd-dpbvp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.11:8443/health\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body=
Jul 28 20:26:07.513826 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.513713    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5b999d9bcd-dpbvp" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerName="console" probeResult="failure" output="Get \"https://10.133.0.11:8443/health\": dial tcp 10.133.0.11:8443: connect: connection refused"
Jul 28 20:26:07.763358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.763273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:07.866233 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:07.866198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:08.754756 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:08.754714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5bbt" event={"ID":"1ece9f1f-8051-4b3b-a516-60adc8b8df00","Type":"ContainerStarted","Data":"a88414e1fccfae898f4dd5f8aacde7b9e321bd836d32d47a594cdd360b2bbd37"}
Jul 28 20:26:08.783981 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:08.783874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:08.892512 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:08.892469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:09.033848 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.033810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:09.134016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.133923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:09.965416 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.965381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:26:09.965839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.965426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:26:09.968114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.968087    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:26:09.968248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.968090    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:26:09.978131 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.978101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3144fb9b-2e19-46b6-ad95-b25777f15fc2-original-pull-secret\") pod \"global-pull-secret-syncer-6746s\" (UID: \"3144fb9b-2e19-46b6-ad95-b25777f15fc2\") " pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:26:09.978279 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:09.978101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a10a7ae8-3062-40b0-8c82-aa7e31c93b7f-metrics-certs\") pod \"network-metrics-daemon-kvbr4\" (UID: \"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f\") " pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:26:10.066291 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.066252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:26:10.069052 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.069030    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 20:26:10.079704 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.079681    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 20:26:10.090907 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.090879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jzmj\" (UniqueName: \"kubernetes.io/projected/664da406-c952-48bf-acd5-577f1f39740d-kube-api-access-6jzmj\") pod \"network-check-target-fb2rf\" (UID: \"664da406-c952-48bf-acd5-577f1f39740d\") " pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:26:10.203645 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.203608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6746s"
Jul 28 20:26:10.210874 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.210666    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sn8zg\""
Jul 28 20:26:10.219860 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.219019    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvbr4"
Jul 28 20:26:10.315989 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.315930    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9c5p9\""
Jul 28 20:26:10.323790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.323548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:26:10.354457 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.354375    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6746s"]
Jul 28 20:26:10.357723 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:10.357694    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3144fb9b_2e19_46b6_ad95_b25777f15fc2.slice/crio-08d22a91b356c5bc5d83d0bc6a4302aeb1b4f35f650b396f100c03a27f256b0b WatchSource:0}: Error finding container 08d22a91b356c5bc5d83d0bc6a4302aeb1b4f35f650b396f100c03a27f256b0b: Status 404 returned error can't find the container with id 08d22a91b356c5bc5d83d0bc6a4302aeb1b4f35f650b396f100c03a27f256b0b
Jul 28 20:26:10.369933 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.369899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvbr4"]
Jul 28 20:26:10.378128 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:10.378090    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10a7ae8_3062_40b0_8c82_aa7e31c93b7f.slice/crio-6b3a8b023b5e85093f704ba75ccc152779ad2f7362a39be26a73ad9d4b5a0778 WatchSource:0}: Error finding container 6b3a8b023b5e85093f704ba75ccc152779ad2f7362a39be26a73ad9d4b5a0778: Status 404 returned error can't find the container with id 6b3a8b023b5e85093f704ba75ccc152779ad2f7362a39be26a73ad9d4b5a0778
Jul 28 20:26:10.473350 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.473276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fb2rf"]
Jul 28 20:26:10.730092 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.730019    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vqjsk"
Jul 28 20:26:10.764327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.764291    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvbr4" event={"ID":"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f","Type":"ContainerStarted","Data":"6b3a8b023b5e85093f704ba75ccc152779ad2f7362a39be26a73ad9d4b5a0778"}
Jul 28 20:26:10.765808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:10.765780    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6746s" event={"ID":"3144fb9b-2e19-46b6-ad95-b25777f15fc2","Type":"ContainerStarted","Data":"08d22a91b356c5bc5d83d0bc6a4302aeb1b4f35f650b396f100c03a27f256b0b"}
Jul 28 20:26:11.545974 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:11.545925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:11.649980 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:11.649941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:12.116077 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:12.116034    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664da406_c952_48bf_acd5_577f1f39740d.slice/crio-434d0b3f9a10c57f32321f926af35c3918358720c1a7e8ef8130cf3b9cd5510d WatchSource:0}: Error finding container 434d0b3f9a10c57f32321f926af35c3918358720c1a7e8ef8130cf3b9cd5510d: Status 404 returned error can't find the container with id 434d0b3f9a10c57f32321f926af35c3918358720c1a7e8ef8130cf3b9cd5510d
Jul 28 20:26:12.776751 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:12.776588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5bbt" event={"ID":"1ece9f1f-8051-4b3b-a516-60adc8b8df00","Type":"ContainerStarted","Data":"06a2b3ddea3842573619dea5f75918d39038619623aa747955a04c7f5c2489a6"}
Jul 28 20:26:12.778619 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:12.778587    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb2rf" event={"ID":"664da406-c952-48bf-acd5-577f1f39740d","Type":"ContainerStarted","Data":"434d0b3f9a10c57f32321f926af35c3918358720c1a7e8ef8130cf3b9cd5510d"}
Jul 28 20:26:12.795086 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:12.794869    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n5bbt" podStartSLOduration=4.652301769 podStartE2EDuration="18.794850119s" totalImagesPullingTime="14.14254835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:12.794274944 +0000 UTC m=+67.979379651" watchObservedRunningTime="2026-07-28 20:26:12.794850119 +0000 UTC m=+67.979954826"
Jul 28 20:26:13.065737 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.065688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:13.135502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.135456    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:13.199508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.199473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:13.785743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.785703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvbr4" event={"ID":"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f","Type":"ContainerStarted","Data":"da358a0e37ae75c880da9c270b3283d52a9fcddb632e0e0ed47556e29c98b2d3"}
Jul 28 20:26:13.786240 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.785770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvbr4" event={"ID":"a10a7ae8-3062-40b0-8c82-aa7e31c93b7f","Type":"ContainerStarted","Data":"0c1887bc54a3200e19a0e9e798fc94710e538ce8152a678a1e64a3776338fff7"}
Jul 28 20:26:13.805377 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.805295    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kvbr4" podStartSLOduration=65.477560697 podStartE2EDuration="1m7.80527409s" totalImagesPullingTime="2.327713393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:13.804396189 +0000 UTC m=+68.989500897" watchObservedRunningTime="2026-07-28 20:26:13.80527409 +0000 UTC m=+68.990378797"
Jul 28 20:26:13.933136 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:13.933103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:14.008129 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:14.008097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:14.076542 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:14.076439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:14.553508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:14.553475    2576 patch_prober.go:28] interesting pod/image-registry-6cdd7ddc4b-ccfjl container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 20:26:14.553684 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:14.553534    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl" podUID="4cc0a2c6-643f-43d4-a19c-75def068ffdf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 20:26:15.031032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.030968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.090011 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.089953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.203163 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.203129    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:15.208193 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.208164    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:15.374047 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.373949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.475283 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.475242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.578125 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.578085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.816911 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.816863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:15.890559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:15.890506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:16.708002 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.707978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cdd7ddc4b-ccfjl"
Jul 28 20:26:16.761088 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.761054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:16.797804 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.797768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6746s" event={"ID":"3144fb9b-2e19-46b6-ad95-b25777f15fc2","Type":"ContainerStarted","Data":"c19e5f2adb5059e9654eccb5ca2b93a3985d25ad5210dc9e77ef47f6cb4c690b"}
Jul 28 20:26:16.799442 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.799418    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fb2rf" event={"ID":"664da406-c952-48bf-acd5-577f1f39740d","Type":"ContainerStarted","Data":"a8f93f91dfd52929e772e146386fa0cefcf1571b3efe1aca9474219c67351cfd"}
Jul 28 20:26:16.799576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.799504    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:26:16.833880 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.833829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6746s" podStartSLOduration=64.527995208 podStartE2EDuration="1m10.833812966s" totalImagesPullingTime="6.305817758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:16.831919368 +0000 UTC m=+72.017024078" watchObservedRunningTime="2026-07-28 20:26:16.833812966 +0000 UTC m=+72.018917759"
Jul 28 20:26:16.856942 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.856890    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fb2rf" podStartSLOduration=66.312973504 podStartE2EDuration="1m10.856872563s" totalImagesPullingTime="4.543899059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:16.856408353 +0000 UTC m=+72.041513060" watchObservedRunningTime="2026-07-28 20:26:16.856872563 +0000 UTC m=+72.041977269"
Jul 28 20:26:16.868105 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.868076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:16.942080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:16.941983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.043039 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.043004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.157982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.157946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.254294 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.254236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.460619 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.460585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.513512 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.513432    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"]
Jul 28 20:26:17.515915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.515898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.516829 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.516808    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:17.520038 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.520017    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 20:26:17.520171 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.520013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 20:26:17.520924 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.520904    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:17.521028 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.520994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 20:26:17.521028 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.521003    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 20:26:17.521028 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.521020    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 20:26:17.521187 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.521044    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 20:26:17.521187 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.521130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f8gnv\""
Jul 28 20:26:17.532115 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.532087    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"]
Jul 28 20:26:17.550818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.550783    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cf9st"]
Jul 28 20:26:17.553017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.552996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.555274 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.555256    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 20:26:17.555378 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.555339    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 20:26:17.555698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.555685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ml6l9\""
Jul 28 20:26:17.555908 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.555895    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 20:26:17.558565 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.558541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.631195 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.631195 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-sys\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.631440 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.631440 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.631440 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631346    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.631440 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-accelerators-collector-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.631978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-metrics-client-ca\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632154    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-wtmp\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s4mf\" (UniqueName: \"kubernetes.io/projected/4a3bc548-7704-4a1b-829d-6910659fba19-kube-api-access-2s4mf\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-root\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfsdb\" (UniqueName: \"kubernetes.io/projected/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-api-access-tfsdb\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-textfile\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.633277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.632524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.639922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.639896    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:26:17.733469 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.733469 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-sys\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-sys\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:26:17.733633    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-accelerators-collector-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:26:17.733713    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls podName:4a3bc548-7704-4a1b-829d-6910659fba19 nodeName:}" failed. No retries permitted until 2026-07-28 20:26:18.233690218 +0000 UTC m=+73.418794919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls") pod "node-exporter-cf9st" (UID: "4a3bc548-7704-4a1b-829d-6910659fba19") : secret "node-exporter-tls" not found
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-metrics-client-ca\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-wtmp\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s4mf\" (UniqueName: \"kubernetes.io/projected/4a3bc548-7704-4a1b-829d-6910659fba19-kube-api-access-2s4mf\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-root\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733913    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfsdb\" (UniqueName: \"kubernetes.io/projected/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-api-access-tfsdb\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.733966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.733970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-textfile\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-wtmp\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-accelerators-collector-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3bc548-7704-4a1b-829d-6910659fba19-metrics-client-ca\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3bc548-7704-4a1b-829d-6910659fba19-root\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.734813 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-textfile\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.735139 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.734986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.735139 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.735031    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.736489 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.736469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.736598 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.736577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.736667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.736649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.750212 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.750188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfsdb\" (UniqueName: \"kubernetes.io/projected/882e3cc3-edc1-4141-8b27-9f5b9d604a9e-kube-api-access-tfsdb\") pod \"kube-state-metrics-56d9f944b4-ssrt2\" (UID: \"882e3cc3-edc1-4141-8b27-9f5b9d604a9e\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.751191 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.751172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s4mf\" (UniqueName: \"kubernetes.io/projected/4a3bc548-7704-4a1b-829d-6910659fba19-kube-api-access-2s4mf\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:17.782693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.782611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.825614 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.825581    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"
Jul 28 20:26:17.888034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.888001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.959122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.959090    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2"]
Jul 28 20:26:17.962012 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:17.961984    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882e3cc3_edc1_4141_8b27_9f5b9d604a9e.slice/crio-513aa17713028c9e31cc6934fa22d573ce2031156f2f2506957860e2fff4fcd4 WatchSource:0}: Error finding container 513aa17713028c9e31cc6934fa22d573ce2031156f2f2506957860e2fff4fcd4: Status 404 returned error can't find the container with id 513aa17713028c9e31cc6934fa22d573ce2031156f2f2506957860e2fff4fcd4
Jul 28 20:26:17.981465 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:17.981434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.082434 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.082321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.235130 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.235099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.239160 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.239140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:18.241420 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.241403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3bc548-7704-4a1b-829d-6910659fba19-node-exporter-tls\") pod \"node-exporter-cf9st\" (UID: \"4a3bc548-7704-4a1b-829d-6910659fba19\") " pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:18.443311 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.443207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.462069 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.462041    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cf9st"
Jul 28 20:26:18.474530 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:18.474505    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3bc548_7704_4a1b_829d_6910659fba19.slice/crio-a5a0a58e80c79660b5ceb8b8bda417a5bbb0802f885dc61d670d923be7efb10a WatchSource:0}: Error finding container a5a0a58e80c79660b5ceb8b8bda417a5bbb0802f885dc61d670d923be7efb10a: Status 404 returned error can't find the container with id a5a0a58e80c79660b5ceb8b8bda417a5bbb0802f885dc61d670d923be7efb10a
Jul 28 20:26:18.532791 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.532757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.754053 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.754019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:18.808000 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.807963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2" event={"ID":"882e3cc3-edc1-4141-8b27-9f5b9d604a9e","Type":"ContainerStarted","Data":"513aa17713028c9e31cc6934fa22d573ce2031156f2f2506957860e2fff4fcd4"}
Jul 28 20:26:18.809293 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:18.809270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cf9st" event={"ID":"4a3bc548-7704-4a1b-829d-6910659fba19","Type":"ContainerStarted","Data":"a5a0a58e80c79660b5ceb8b8bda417a5bbb0802f885dc61d670d923be7efb10a"}
Jul 28 20:26:19.816399 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.816346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2" event={"ID":"882e3cc3-edc1-4141-8b27-9f5b9d604a9e","Type":"ContainerStarted","Data":"6849e53bab4ecde6dcdc834269044e9df56e2c6854454a15dbadb9bf4b285938"}
Jul 28 20:26:19.816885 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.816433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2" event={"ID":"882e3cc3-edc1-4141-8b27-9f5b9d604a9e","Type":"ContainerStarted","Data":"f7f1e32b1df268b35258d84065637795a44e4825ba81144429e31d2ee54ff8bd"}
Jul 28 20:26:19.816885 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.816463    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2" event={"ID":"882e3cc3-edc1-4141-8b27-9f5b9d604a9e","Type":"ContainerStarted","Data":"bb95b68843ca22b46bcb49ca8f638e326cb9a20188a30c7584225bda94b84167"}
Jul 28 20:26:19.840218 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.840162    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-ssrt2" podStartSLOduration=1.369134536 podStartE2EDuration="2.84014405s" totalImagesPullingTime="1.471009514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:19.837387722 +0000 UTC m=+75.022492631" watchObservedRunningTime="2026-07-28 20:26:19.84014405 +0000 UTC m=+75.025248756"
Jul 28 20:26:19.868312 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.867934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:19.967922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:19.967873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.143504 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.143467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.237318 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.237283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.422831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.422742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.484652 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.484620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.758923 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.758886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:20.825143 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.825103    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a3bc548-7704-4a1b-829d-6910659fba19" containerID="e25057a6dc9d5dfd9bff32668d638c148f10a918e7e96ecaff66db41ffae60c8" exitCode=0
Jul 28 20:26:20.825581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.825193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cf9st" event={"ID":"4a3bc548-7704-4a1b-829d-6910659fba19","Type":"ContainerDied","Data":"e25057a6dc9d5dfd9bff32668d638c148f10a918e7e96ecaff66db41ffae60c8"}
Jul 28 20:26:20.915116 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:20.915088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.814941 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.814909    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"]
Jul 28 20:26:21.817033 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.817013    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.819281 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.819263    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 20:26:21.820264 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.820246    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 20:26:21.820492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.820448    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 20:26:21.820492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.820467    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dfvvm\""
Jul 28 20:26:21.820633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.820497    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 20:26:21.820633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.820562    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cua8fjsgh1bcj\""
Jul 28 20:26:21.830272 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.830244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cf9st" event={"ID":"4a3bc548-7704-4a1b-829d-6910659fba19","Type":"ContainerStarted","Data":"e49a65865a9659bf47db8738970845148c6f36cbf2667389ef36d38fdf37b314"}
Jul 28 20:26:21.830648 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.830301    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cf9st" event={"ID":"4a3bc548-7704-4a1b-829d-6910659fba19","Type":"ContainerStarted","Data":"45492e1582d401caeea5a5b4e61407c9f9c96aef5e20eabf586d911d41f1817c"}
Jul 28 20:26:21.831673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.831653    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"]
Jul 28 20:26:21.862443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.862390    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cf9st" podStartSLOduration=3.280840347 podStartE2EDuration="4.862349768s" totalImagesPullingTime="1.517424486s" totalInitContainerRuntime="64.084935ms" isStatefulPod=true podCreationTimestamp="2026-07-28 20:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:21.861145943 +0000 UTC m=+77.046250650" watchObservedRunningTime="2026-07-28 20:26:21.862349768 +0000 UTC m=+77.047454474"
Jul 28 20:26:21.958610 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.958570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:21.972620 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.972588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-client-certs\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.972870 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.972807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-client-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.972983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.972871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-tls\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.972983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.972907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.973091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.972998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-audit-log\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.973091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.973071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-metrics-server-audit-profiles\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:21.973196 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:21.973100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8dfp\" (UniqueName: \"kubernetes.io/projected/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-kube-api-access-f8dfp\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.073917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-client-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.073969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-tls\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.073999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-audit-log\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-metrics-server-audit-profiles\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8dfp\" (UniqueName: \"kubernetes.io/projected/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-kube-api-access-f8dfp\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-client-certs\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074610 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-audit-log\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.074966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.074941    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.075111 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.075094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-metrics-server-audit-profiles\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.076536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.076512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-client-ca-bundle\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.076624 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.076551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-tls\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.076624 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.076559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-secret-metrics-server-client-certs\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.093671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.093648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8dfp\" (UniqueName: \"kubernetes.io/projected/e90ca0a3-0b1d-4f18-abc8-9e654c8afe61-kube-api-access-f8dfp\") pod \"metrics-server-bdcf88c4d-jjq5x\" (UID: \"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61\") " pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.109021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.108990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.126032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.125998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:22.260716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.260681    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"]
Jul 28 20:26:22.263275 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:22.263249    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode90ca0a3_0b1d_4f18_abc8_9e654c8afe61.slice/crio-9507ddfba0e127b4790db7122260eb1e88f17e315e368400077ae221828a9ec3 WatchSource:0}: Error finding container 9507ddfba0e127b4790db7122260eb1e88f17e315e368400077ae221828a9ec3: Status 404 returned error can't find the container with id 9507ddfba0e127b4790db7122260eb1e88f17e315e368400077ae221828a9ec3
Jul 28 20:26:22.265919 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.265896    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:26:22.267959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.267934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.268762 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.268745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.282659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.282637    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:26:22.344356 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.344269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.376321 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376484    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhf9\" (UniqueName: \"kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.376688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.376534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.445207 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.445174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.477195 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477455 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhf9\" (UniqueName: \"kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477455 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477455 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.477977 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.477935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.478116 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.478094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.478167 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.478120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.478243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.478227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.479697 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.479673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.479823 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.479806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.488415 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.488396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhf9\" (UniqueName: \"kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9\") pod \"console-74b5f56cc-kgvsn\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") " pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.505750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.505716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:22.578044 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.578000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:22.735428 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.735396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:26:22.738129 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:26:22.738105    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42578370_249d_4328_9cd3_7562e007f893.slice/crio-6051f83d1b926407dd948e0c14e410b2797ff75e336cafef1b79999a0899a287 WatchSource:0}: Error finding container 6051f83d1b926407dd948e0c14e410b2797ff75e336cafef1b79999a0899a287: Status 404 returned error can't find the container with id 6051f83d1b926407dd948e0c14e410b2797ff75e336cafef1b79999a0899a287
Jul 28 20:26:22.840829 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.840786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b5f56cc-kgvsn" event={"ID":"42578370-249d-4328-9cd3-7562e007f893","Type":"ContainerStarted","Data":"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"}
Jul 28 20:26:22.841305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.840867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b5f56cc-kgvsn" event={"ID":"42578370-249d-4328-9cd3-7562e007f893","Type":"ContainerStarted","Data":"6051f83d1b926407dd948e0c14e410b2797ff75e336cafef1b79999a0899a287"}
Jul 28 20:26:22.842141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.842106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x" event={"ID":"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61","Type":"ContainerStarted","Data":"9507ddfba0e127b4790db7122260eb1e88f17e315e368400077ae221828a9ec3"}
Jul 28 20:26:22.858912 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:22.858863    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74b5f56cc-kgvsn" podStartSLOduration=0.85884536 podStartE2EDuration="858.84536ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:22.858113183 +0000 UTC m=+78.043217902" watchObservedRunningTime="2026-07-28 20:26:22.85884536 +0000 UTC m=+78.043950065"
Jul 28 20:26:23.711064 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:23.711026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:23.848261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:23.848218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x" event={"ID":"e90ca0a3-0b1d-4f18-abc8-9e654c8afe61","Type":"ContainerStarted","Data":"f41e012e6f191f0ef65de524a598fee56a880d965703c0f6241acdc90e827d69"}
Jul 28 20:26:23.866636 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:23.866591    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x" podStartSLOduration=1.374833204 podStartE2EDuration="2.866575653s" totalImagesPullingTime="1.491742449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:23.866028045 +0000 UTC m=+79.051132751" watchObservedRunningTime="2026-07-28 20:26:23.866575653 +0000 UTC m=+79.051680359"
Jul 28 20:26:25.134872 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:25.134836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:25.235032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:25.234993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:25.810278 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:25.810239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:25.905059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:25.905026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:27.713160 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:27.713039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:27.817261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:27.817227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:28.873498 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:28.873451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:28.970951 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:28.970915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:31.076297 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:31.076259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:31.176292 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:31.176258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:32.579079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:32.579037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:32.579551 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:32.579100    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:32.583959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:32.583933    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:32.882074 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:32.881998    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:26:32.927492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:32.927459    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:26:33.022905 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:33.022867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:33.125870 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:33.125831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:33.179998 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:33.179919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.126409 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.126353    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:42.126803 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.126439    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:26:42.658856 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.658808    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cf8bfd944-sgwhf" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerName="console" containerID="cri-o://0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f" gracePeriod=15
Jul 28 20:26:42.717190 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.717153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.787211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.787175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.891177 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.891152    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:42.913356 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.913289    2576 generic.go:356] "Generic (PLEG): container finished" podID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerID="0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f" exitCode=0
Jul 28 20:26:42.913356 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.913326    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf8bfd944-sgwhf"
Jul 28 20:26:42.913528 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.913448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf8bfd944-sgwhf" event={"ID":"c58ded32-6d98-46f6-b4a0-156d70c69f2c","Type":"ContainerDied","Data":"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"}
Jul 28 20:26:42.913528 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.913511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf8bfd944-sgwhf" event={"ID":"c58ded32-6d98-46f6-b4a0-156d70c69f2c","Type":"ContainerDied","Data":"0e1aa02dab05e06f77e5eb49b75c468c0b528dddbb8a893a4a1541b4891f60c0"}
Jul 28 20:26:42.913595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.913548    2576 scope.go:117] "RemoveContainer" containerID="0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"
Jul 28 20:26:42.923231 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.923214    2576 scope.go:117] "RemoveContainer" containerID="0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"
Jul 28 20:26:42.923547 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:26:42.923517    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f\": container with ID starting with 0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f not found: ID does not exist" containerID="0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"
Jul 28 20:26:42.923607 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.923560    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f"} err="failed to get container status \"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f\": rpc error: code = NotFound desc = could not find container \"0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f\": container with ID starting with 0b08efe5371643c56ff36d10385023f6d51732a1ec1323d1d33b74b0f294908f not found: ID does not exist"
Jul 28 20:26:42.945957 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.945927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.945980    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztdnq\" (UniqueName: \"kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946090    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert\") pod \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\" (UID: \"c58ded32-6d98-46f6-b4a0-156d70c69f2c\") "
Jul 28 20:26:42.946529 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config" (OuterVolumeSpecName: "console-config") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:42.946645 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946538    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:42.946645 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.946414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca" (OuterVolumeSpecName: "service-ca") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:42.948462 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.948437    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq" (OuterVolumeSpecName: "kube-api-access-ztdnq") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "kube-api-access-ztdnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:42.948694 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.948671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:26:42.948761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:42.948726    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c58ded32-6d98-46f6-b4a0-156d70c69f2c" (UID: "c58ded32-6d98-46f6-b4a0-156d70c69f2c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:26:43.047357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047315    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.047357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047349    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.047357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047388    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztdnq\" (UniqueName: \"kubernetes.io/projected/c58ded32-6d98-46f6-b4a0-156d70c69f2c-kube-api-access-ztdnq\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.047621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047402    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.047621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047414    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c58ded32-6d98-46f6-b4a0-156d70c69f2c-oauth-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.047621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.047425    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c58ded32-6d98-46f6-b4a0-156d70c69f2c-console-oauth-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:43.239222 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.239188    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:26:43.243126 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.243100    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cf8bfd944-sgwhf"]
Jul 28 20:26:43.298793 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:43.298761    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" path="/var/lib/kubelet/pods/c58ded32-6d98-46f6-b4a0-156d70c69f2c/volumes"
Jul 28 20:26:47.805689 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:47.805655    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fb2rf"
Jul 28 20:26:51.851348 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:51.851191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:51.956517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:51.956479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:57.948270 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:57.948205    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b999d9bcd-dpbvp" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerName="console" containerID="cri-o://393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff" gracePeriod=15
Jul 28 20:26:58.182226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.182203    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:58.248480 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzd88\" (UniqueName: \"kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248490    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248513    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248625    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.248875 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248658    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert\") pod \"7e89c85b-dbfc-4805-978d-0be5351d6841\" (UID: \"7e89c85b-dbfc-4805-978d-0be5351d6841\") "
Jul 28 20:26:58.249038 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:58.249165 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.248976    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config" (OuterVolumeSpecName: "console-config") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:58.249165 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.249113    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca" (OuterVolumeSpecName: "service-ca") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:58.249270 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.249177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:26:58.250839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.250801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:26:58.250839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.250829    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88" (OuterVolumeSpecName: "kube-api-access-pzd88") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "kube-api-access-pzd88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:26:58.250967 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.250849    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7e89c85b-dbfc-4805-978d-0be5351d6841" (UID: "7e89c85b-dbfc-4805-978d-0be5351d6841"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:26:58.350079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350029    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-oauth-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350074    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350084    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-oauth-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350093    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzd88\" (UniqueName: \"kubernetes.io/projected/7e89c85b-dbfc-4805-978d-0be5351d6841-kube-api-access-pzd88\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350350 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350103    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-trusted-ca-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350350 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350112    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e89c85b-dbfc-4805-978d-0be5351d6841-console-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.350350 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.350120    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e89c85b-dbfc-4805-978d-0be5351d6841-console-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:26:58.969135 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.969093    2576 generic.go:356] "Generic (PLEG): container finished" podID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerID="393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff" exitCode=0
Jul 28 20:26:58.969135 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.969141    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b999d9bcd-dpbvp"
Jul 28 20:26:58.969695 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.969140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b999d9bcd-dpbvp" event={"ID":"7e89c85b-dbfc-4805-978d-0be5351d6841","Type":"ContainerDied","Data":"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"}
Jul 28 20:26:58.969695 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.969263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b999d9bcd-dpbvp" event={"ID":"7e89c85b-dbfc-4805-978d-0be5351d6841","Type":"ContainerDied","Data":"a78513ad5ca8f9b8b52cbbcdf57bc4b11233e5ccdfefb548dc4cd1e4e7446bb0"}
Jul 28 20:26:58.969695 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.969290    2576 scope.go:117] "RemoveContainer" containerID="393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"
Jul 28 20:26:58.977403 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.977385    2576 scope.go:117] "RemoveContainer" containerID="393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"
Jul 28 20:26:58.977631 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:26:58.977613    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff\": container with ID starting with 393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff not found: ID does not exist" containerID="393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"
Jul 28 20:26:58.977676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.977639    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff"} err="failed to get container status \"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff\": rpc error: code = NotFound desc = could not find container \"393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff\": container with ID starting with 393f29fb6565db418b66daffa6517529eaef15a371b08ad5e0292854d84c69ff not found: ID does not exist"
Jul 28 20:26:58.991323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:58.991297    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:26:59.006926 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:59.006904    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b999d9bcd-dpbvp"]
Jul 28 20:26:59.298320 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:26:59.298289    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" path="/var/lib/kubelet/pods/7e89c85b-dbfc-4805-978d-0be5351d6841/volumes"
Jul 28 20:27:02.134659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:02.134619    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:27:02.138787 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:02.138757    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bdcf88c4d-jjq5x"
Jul 28 20:27:33.649927 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:33.649896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:36.744858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:36.744817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:37.145978 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:37.145885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:41.836800 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.836766    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"]
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837107    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerName="console"
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837124    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerName="console"
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837149    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerName="console"
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837158    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerName="console"
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837212    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e89c85b-dbfc-4805-978d-0be5351d6841" containerName="console"
Jul 28 20:27:41.837405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.837223    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c58ded32-6d98-46f6-b4a0-156d70c69f2c" containerName="console"
Jul 28 20:27:41.839289 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.839269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.845694 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845665    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vldl8\""
Jul 28 20:27:41.845808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845717    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 20:27:41.845808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845776    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 20:27:41.845808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845777    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 20:27:41.845993 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845813    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 20:27:41.845993 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.845988    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 20:27:41.850581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.850563    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 20:27:41.852767 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.852744    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"]
Jul 28 20:27:41.979812 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.979812 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-federate-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-serving-certs-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpb8h\" (UniqueName: \"kubernetes.io/projected/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-kube-api-access-rpb8h\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-metrics-client-ca\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:41.980032 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:41.979987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080731 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpb8h\" (UniqueName: \"kubernetes.io/projected/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-kube-api-access-rpb8h\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080810    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-metrics-client-ca\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.080922 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.080873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.081198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.081002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-federate-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.081198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.081062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-serving-certs-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.081665 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.081636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-metrics-client-ca\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.081805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.081783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-serving-certs-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.081980 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.081956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.083358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.083329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.083547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.083526    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.083547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.083538    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-federate-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.083622 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.083578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-telemeter-client-tls\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.089714 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.089649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpb8h\" (UniqueName: \"kubernetes.io/projected/76bd763a-bb1d-4bfc-81f7-6553ae63b6f3-kube-api-access-rpb8h\") pod \"telemeter-client-5f566ddcb6-mz56w\" (UID: \"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3\") " pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.148501 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.148471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"
Jul 28 20:27:42.280315 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:42.280273    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f566ddcb6-mz56w"]
Jul 28 20:27:42.284453 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:27:42.284424    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76bd763a_bb1d_4bfc_81f7_6553ae63b6f3.slice/crio-8e1ed88e10fe3d720183bbd3ea9091d0d4b1875bd133504e7e1de23f5ed69794 WatchSource:0}: Error finding container 8e1ed88e10fe3d720183bbd3ea9091d0d4b1875bd133504e7e1de23f5ed69794: Status 404 returned error can't find the container with id 8e1ed88e10fe3d720183bbd3ea9091d0d4b1875bd133504e7e1de23f5ed69794
Jul 28 20:27:43.117241 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:43.117199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w" event={"ID":"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3","Type":"ContainerStarted","Data":"8e1ed88e10fe3d720183bbd3ea9091d0d4b1875bd133504e7e1de23f5ed69794"}
Jul 28 20:27:44.121693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:44.121608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w" event={"ID":"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3","Type":"ContainerStarted","Data":"c1cba099ae0699120be32a4e00b1983ccef6931efe314bb3e4a287c766659275"}
Jul 28 20:27:46.129808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:46.129762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w" event={"ID":"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3","Type":"ContainerStarted","Data":"c3cbc117ff9e5a4c73cf30880c87591e3c8d83331a27332b0bb477c7d052173c"}
Jul 28 20:27:46.130225 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:46.129823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w" event={"ID":"76bd763a-bb1d-4bfc-81f7-6553ae63b6f3","Type":"ContainerStarted","Data":"0045bdf2edd34717ab204afcfc3ab4faf33a9884586281acb9d4502dd446ecdb"}
Jul 28 20:27:46.155004 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:46.154951    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f566ddcb6-mz56w" podStartSLOduration=2.045866189 podStartE2EDuration="5.154937054s" totalImagesPullingTime="3.109070865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:46.152520331 +0000 UTC m=+161.337625037" watchObservedRunningTime="2026-07-28 20:27:46.154937054 +0000 UTC m=+161.340041760"
Jul 28 20:27:47.259990 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.259954    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:27:47.262153 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.262125    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.274920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.274892    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:27:47.327004 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.326967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:47.397438 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.397395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:47.427360 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427360 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlc2g\" (UniqueName: \"kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.427596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.427596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528389 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528389 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528389 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528423    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.528662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.528478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlc2g\" (UniqueName: \"kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.529185 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.529159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.529291 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.529251    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.529454 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.529428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.529690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.529672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.530893 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.530871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.531000 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.530966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.537156 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.537135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlc2g\" (UniqueName: \"kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g\") pod \"console-788cff987-fptjb\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") " pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.571083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.571042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:47.696707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:47.696683    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:27:47.699039 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:27:47.699004    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a60c77_1a08_4887_8890_4fbd22d40b2a.slice/crio-8192bf55e829a8afa3739aeaaf5957224c15ab679559b28cb51da07934c4a082 WatchSource:0}: Error finding container 8192bf55e829a8afa3739aeaaf5957224c15ab679559b28cb51da07934c4a082: Status 404 returned error can't find the container with id 8192bf55e829a8afa3739aeaaf5957224c15ab679559b28cb51da07934c4a082
Jul 28 20:27:48.137254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:48.137216    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788cff987-fptjb" event={"ID":"e7a60c77-1a08-4887-8890-4fbd22d40b2a","Type":"ContainerStarted","Data":"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"}
Jul 28 20:27:48.137466 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:48.137265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788cff987-fptjb" event={"ID":"e7a60c77-1a08-4887-8890-4fbd22d40b2a","Type":"ContainerStarted","Data":"8192bf55e829a8afa3739aeaaf5957224c15ab679559b28cb51da07934c4a082"}
Jul 28 20:27:52.002839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:52.002801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:52.063067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:52.063026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:57.571419 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:57.571316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:57.571419 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:57.571410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:57.576034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:57.576010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:57.595674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:57.595412    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-788cff987-fptjb" podStartSLOduration=10.595392534 podStartE2EDuration="10.595392534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:48.159341515 +0000 UTC m=+163.344446222" watchObservedRunningTime="2026-07-28 20:27:57.595392534 +0000 UTC m=+172.780497241"
Jul 28 20:27:58.175663 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.175635    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:27:58.256879 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.256842    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:27:58.381866 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.381829    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-587465f6b9-smp5n"]
Jul 28 20:27:58.385407 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.385389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.390458 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.390429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:58.395894 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.395872    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587465f6b9-smp5n"]
Jul 28 20:27:58.401707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqmhg\" (UniqueName: \"kubernetes.io/projected/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-kube-api-access-fqmhg\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-oauth-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-service-ca\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401967 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-oauth-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401967 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-trusted-ca-bundle\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.401967 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.401943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.456719 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.456627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:58.502718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-oauth-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-trusted-ca-bundle\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqmhg\" (UniqueName: \"kubernetes.io/projected/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-kube-api-access-fqmhg\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-oauth-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.502920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.502914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-service-ca\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.503554 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.503529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-oauth-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.503685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.503578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.503685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.503677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-service-ca\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.504020 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.503999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-trusted-ca-bundle\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.505472 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.505449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-oauth-config\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.505528 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.505477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-console-serving-cert\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.511454 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.511430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqmhg\" (UniqueName: \"kubernetes.io/projected/a07cde6c-9eca-4f3b-9789-8b09e05bf3a0-kube-api-access-fqmhg\") pod \"console-587465f6b9-smp5n\" (UID: \"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0\") " pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.695067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.695028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:27:58.712955 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.712925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:58.771323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.771291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:58.822983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:58.822957    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-587465f6b9-smp5n"]
Jul 28 20:27:58.825286 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:27:58.825257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07cde6c_9eca_4f3b_9789_8b09e05bf3a0.slice/crio-3ddea9e3e3642dd8fc36eca528d8d2238b957e75fec6c6022674376b8e4281e6 WatchSource:0}: Error finding container 3ddea9e3e3642dd8fc36eca528d8d2238b957e75fec6c6022674376b8e4281e6: Status 404 returned error can't find the container with id 3ddea9e3e3642dd8fc36eca528d8d2238b957e75fec6c6022674376b8e4281e6
Jul 28 20:27:59.176777 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:59.176738    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587465f6b9-smp5n" event={"ID":"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0","Type":"ContainerStarted","Data":"9c0958dd53fda0bda7e823923338364314e97455d49a1765a913f37b092eae12"}
Jul 28 20:27:59.176959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:59.176801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-587465f6b9-smp5n" event={"ID":"a07cde6c-9eca-4f3b-9789-8b09e05bf3a0","Type":"ContainerStarted","Data":"3ddea9e3e3642dd8fc36eca528d8d2238b957e75fec6c6022674376b8e4281e6"}
Jul 28 20:27:59.195583 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:27:59.195535    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-587465f6b9-smp5n" podStartSLOduration=1.195517946 podStartE2EDuration="1.195517946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:59.194496672 +0000 UTC m=+174.379601379" watchObservedRunningTime="2026-07-28 20:27:59.195517946 +0000 UTC m=+174.380622651"
Jul 28 20:28:04.007973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:04.007928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:04.085697 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:04.085654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:08.695751 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:08.695700    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:28:08.695751 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:08.695769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:28:08.700299 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:08.700278    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:28:09.216778 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:09.216749    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-587465f6b9-smp5n"
Jul 28 20:28:09.263065 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:09.263031    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:28:09.389873 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:09.389840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:09.490216 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:09.490184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:09.568973 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:09.568937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:12.726846 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:12.726806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:12.826560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:12.826528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:22.130604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:22.130561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:22.235254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:22.235176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:23.276080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.276025    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74b5f56cc-kgvsn" podUID="42578370-249d-4328-9cd3-7562e007f893" containerName="console" containerID="cri-o://848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291" gracePeriod=15
Jul 28 20:28:23.520674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.520649    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:28:23.609090 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.608994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609090 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609056    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609090 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609080    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609353 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609201    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609353 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609353 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xhf9\" (UniqueName: \"kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609353 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config\") pod \"42578370-249d-4328-9cd3-7562e007f893\" (UID: \"42578370-249d-4328-9cd3-7562e007f893\") "
Jul 28 20:28:23.609631 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609541    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:23.609631 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609578    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config" (OuterVolumeSpecName: "console-config") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:23.609631 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609627    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-trusted-ca-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.609768 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609632    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca" (OuterVolumeSpecName: "service-ca") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:23.609768 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.609691    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:23.611334 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.611299    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:28:23.611532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.611359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9" (OuterVolumeSpecName: "kube-api-access-5xhf9") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "kube-api-access-5xhf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:28:23.611532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.611486    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "42578370-249d-4328-9cd3-7562e007f893" (UID: "42578370-249d-4328-9cd3-7562e007f893"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:28:23.710306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710266    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.710306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710300    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-oauth-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.710306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710309    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xhf9\" (UniqueName: \"kubernetes.io/projected/42578370-249d-4328-9cd3-7562e007f893-kube-api-access-5xhf9\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.710306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710320    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-oauth-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.710597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710329    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/42578370-249d-4328-9cd3-7562e007f893-console-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:23.710597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:23.710338    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/42578370-249d-4328-9cd3-7562e007f893-console-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:24.265555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.265519    2576 generic.go:356] "Generic (PLEG): container finished" podID="42578370-249d-4328-9cd3-7562e007f893" containerID="848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291" exitCode=0
Jul 28 20:28:24.265739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.265566    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b5f56cc-kgvsn"
Jul 28 20:28:24.265739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.265580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b5f56cc-kgvsn" event={"ID":"42578370-249d-4328-9cd3-7562e007f893","Type":"ContainerDied","Data":"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"}
Jul 28 20:28:24.265739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.265630    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b5f56cc-kgvsn" event={"ID":"42578370-249d-4328-9cd3-7562e007f893","Type":"ContainerDied","Data":"6051f83d1b926407dd948e0c14e410b2797ff75e336cafef1b79999a0899a287"}
Jul 28 20:28:24.265739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.265658    2576 scope.go:117] "RemoveContainer" containerID="848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"
Jul 28 20:28:24.277700 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.277585    2576 scope.go:117] "RemoveContainer" containerID="848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"
Jul 28 20:28:24.277933 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:28:24.277860    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291\": container with ID starting with 848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291 not found: ID does not exist" containerID="848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"
Jul 28 20:28:24.277933 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.277885    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291"} err="failed to get container status \"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291\": rpc error: code = NotFound desc = could not find container \"848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291\": container with ID starting with 848fb8c8cf655fc80ed877816abe3dc84d7d7975e1490ce53c4d710399adb291 not found: ID does not exist"
Jul 28 20:28:24.294735 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.294702    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:28:24.305072 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:24.305039    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74b5f56cc-kgvsn"]
Jul 28 20:28:25.298808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:25.298775    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42578370-249d-4328-9cd3-7562e007f893" path="/var/lib/kubelet/pods/42578370-249d-4328-9cd3-7562e007f893/volumes"
Jul 28 20:28:34.281830 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.281784    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-788cff987-fptjb" podUID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" containerName="console" containerID="cri-o://3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915" gracePeriod=15
Jul 28 20:28:34.525107 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.525079    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:28:34.701133 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701040    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701133 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701156    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701217    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701319    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701589 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701411    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlc2g\" (UniqueName: \"kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g\") pod \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\" (UID: \"e7a60c77-1a08-4887-8890-4fbd22d40b2a\") "
Jul 28 20:28:34.701589 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701487    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config" (OuterVolumeSpecName: "console-config") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:34.701696 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701494    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:34.701696 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701568    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:34.701696 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701661    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca" (OuterVolumeSpecName: "service-ca") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:28:34.701830 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701707    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.701830 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701720    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-trusted-ca-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.701830 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.701729    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-oauth-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.703459 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.703426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:28:34.703553 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.703479    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:28:34.703553 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.703493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g" (OuterVolumeSpecName: "kube-api-access-nlc2g") pod "e7a60c77-1a08-4887-8890-4fbd22d40b2a" (UID: "e7a60c77-1a08-4887-8890-4fbd22d40b2a"). InnerVolumeSpecName "kube-api-access-nlc2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:28:34.802674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.802633    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7a60c77-1a08-4887-8890-4fbd22d40b2a-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.802674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.802666    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlc2g\" (UniqueName: \"kubernetes.io/projected/e7a60c77-1a08-4887-8890-4fbd22d40b2a-kube-api-access-nlc2g\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.802674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.802676    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-oauth-config\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:34.802970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:34.802687    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a60c77-1a08-4887-8890-4fbd22d40b2a-console-serving-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:28:35.303435 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.303403    2576 generic.go:356] "Generic (PLEG): container finished" podID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" containerID="3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915" exitCode=0
Jul 28 20:28:35.303854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.303449    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788cff987-fptjb"
Jul 28 20:28:35.303854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.303492    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788cff987-fptjb" event={"ID":"e7a60c77-1a08-4887-8890-4fbd22d40b2a","Type":"ContainerDied","Data":"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"}
Jul 28 20:28:35.303854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.303540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788cff987-fptjb" event={"ID":"e7a60c77-1a08-4887-8890-4fbd22d40b2a","Type":"ContainerDied","Data":"8192bf55e829a8afa3739aeaaf5957224c15ab679559b28cb51da07934c4a082"}
Jul 28 20:28:35.303854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.303560    2576 scope.go:117] "RemoveContainer" containerID="3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"
Jul 28 20:28:35.311335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.311318    2576 scope.go:117] "RemoveContainer" containerID="3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"
Jul 28 20:28:35.311620 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:28:35.311602    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915\": container with ID starting with 3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915 not found: ID does not exist" containerID="3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"
Jul 28 20:28:35.311677 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.311631    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915"} err="failed to get container status \"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915\": rpc error: code = NotFound desc = could not find container \"3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915\": container with ID starting with 3a92f56189255b7faa1cb5aa30bd473f61844be2ddd491fb11afa309a22d5915 not found: ID does not exist"
Jul 28 20:28:35.322931 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.322905    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:28:35.327910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:35.327890    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-788cff987-fptjb"]
Jul 28 20:28:37.299319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:37.299281    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" path="/var/lib/kubelet/pods/e7a60c77-1a08-4887-8890-4fbd22d40b2a/volumes"
Jul 28 20:28:45.980949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:45.980915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:46.043002 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:46.042968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:50.365380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:50.365324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:50.466749 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:50.466708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:51.320310 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320278    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"]
Jul 28 20:28:51.320548 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320536    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42578370-249d-4328-9cd3-7562e007f893" containerName="console"
Jul 28 20:28:51.320595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320550    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42578370-249d-4328-9cd3-7562e007f893" containerName="console"
Jul 28 20:28:51.320595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320559    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" containerName="console"
Jul 28 20:28:51.320595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320565    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" containerName="console"
Jul 28 20:28:51.320691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320604    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a60c77-1a08-4887-8890-4fbd22d40b2a" containerName="console"
Jul 28 20:28:51.320691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.320613    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42578370-249d-4328-9cd3-7562e007f893" containerName="console"
Jul 28 20:28:51.325025 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.325009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.327671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.327652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dgzk9\""
Jul 28 20:28:51.327758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.327731    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:28:51.328520 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.328503    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:28:51.334917 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.334896    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"]
Jul 28 20:28:51.434182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.434150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v2ls\" (UniqueName: \"kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.434182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.434188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.434603 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.434250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.534931 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.534888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.535048 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.534955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v2ls\" (UniqueName: \"kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.535048 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.534983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.535341 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.535325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.535407 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.535322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.544075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.544054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v2ls\" (UniqueName: \"kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.634400 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.634287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:28:51.759844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:51.759820    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"]
Jul 28 20:28:51.762455 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:28:51.762421    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60258617_ed49_40d7_8f35_86b25eec2406.slice/crio-825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b WatchSource:0}: Error finding container 825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b: Status 404 returned error can't find the container with id 825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b
Jul 28 20:28:52.364920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:52.364877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s" event={"ID":"60258617-ed49-40d7-8f35-86b25eec2406","Type":"ContainerStarted","Data":"825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b"}
Jul 28 20:28:59.391980 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:59.391943    2576 generic.go:356] "Generic (PLEG): container finished" podID="60258617-ed49-40d7-8f35-86b25eec2406" containerID="979d86ebd2d6a58d98f327d6bf052ea7f868a8fb88484f99aa39fc12f84c5543" exitCode=0
Jul 28 20:28:59.392465 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:28:59.392035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s" event={"ID":"60258617-ed49-40d7-8f35-86b25eec2406","Type":"ContainerDied","Data":"979d86ebd2d6a58d98f327d6bf052ea7f868a8fb88484f99aa39fc12f84c5543"}
Jul 28 20:29:02.405213 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:02.405172    2576 generic.go:356] "Generic (PLEG): container finished" podID="60258617-ed49-40d7-8f35-86b25eec2406" containerID="49c6b571bc4eea29b123f93f79ca76ab5d1341e3bfcee50c755dbae99def0afa" exitCode=0
Jul 28 20:29:02.405622 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:02.405233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s" event={"ID":"60258617-ed49-40d7-8f35-86b25eec2406","Type":"ContainerDied","Data":"49c6b571bc4eea29b123f93f79ca76ab5d1341e3bfcee50c755dbae99def0afa"}
Jul 28 20:29:10.435466 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:10.435423    2576 generic.go:356] "Generic (PLEG): container finished" podID="60258617-ed49-40d7-8f35-86b25eec2406" containerID="76d098f8d84c0f70451a1aeaad7d01b91478a54312cc042c98708e9095db94de" exitCode=0
Jul 28 20:29:10.435873 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:10.435513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s" event={"ID":"60258617-ed49-40d7-8f35-86b25eec2406","Type":"ContainerDied","Data":"76d098f8d84c0f70451a1aeaad7d01b91478a54312cc042c98708e9095db94de"}
Jul 28 20:29:11.604894 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.604861    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:29:11.703402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.703297    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v2ls\" (UniqueName: \"kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls\") pod \"60258617-ed49-40d7-8f35-86b25eec2406\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") "
Jul 28 20:29:11.703402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.703393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle\") pod \"60258617-ed49-40d7-8f35-86b25eec2406\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") "
Jul 28 20:29:11.703630 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.703427    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util\") pod \"60258617-ed49-40d7-8f35-86b25eec2406\" (UID: \"60258617-ed49-40d7-8f35-86b25eec2406\") "
Jul 28 20:29:11.704213 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.704177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle" (OuterVolumeSpecName: "bundle") pod "60258617-ed49-40d7-8f35-86b25eec2406" (UID: "60258617-ed49-40d7-8f35-86b25eec2406"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:11.705576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.705556    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls" (OuterVolumeSpecName: "kube-api-access-5v2ls") pod "60258617-ed49-40d7-8f35-86b25eec2406" (UID: "60258617-ed49-40d7-8f35-86b25eec2406"). InnerVolumeSpecName "kube-api-access-5v2ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:11.708715 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.708692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util" (OuterVolumeSpecName: "util") pod "60258617-ed49-40d7-8f35-86b25eec2406" (UID: "60258617-ed49-40d7-8f35-86b25eec2406"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:11.804121 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.804068    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v2ls\" (UniqueName: \"kubernetes.io/projected/60258617-ed49-40d7-8f35-86b25eec2406-kube-api-access-5v2ls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:11.804121 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.804115    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:11.804121 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:11.804125    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60258617-ed49-40d7-8f35-86b25eec2406-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:12.444527 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:12.444494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s" event={"ID":"60258617-ed49-40d7-8f35-86b25eec2406","Type":"ContainerDied","Data":"825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b"}
Jul 28 20:29:12.444527 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:12.444514    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7f4s"
Jul 28 20:29:12.444738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:12.444541    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="825567e79f4c030963d26b5ba6b568f67d468705c50d1fed6e8520b8bdbc525b"
Jul 28 20:29:17.416593 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:17.416560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:18.297730 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:18.297689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:19.567651 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567616    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"]
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567866    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="extract"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567877    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="extract"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567890    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="pull"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567896    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="pull"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567904    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="util"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567910    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="util"
Jul 28 20:29:19.568096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.567944    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="60258617-ed49-40d7-8f35-86b25eec2406" containerName="extract"
Jul 28 20:29:19.570999 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.570982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.579490 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.579469    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 20:29:19.579595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.579547    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-h9rll\""
Jul 28 20:29:19.579660 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.579638    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:29:19.592243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.592216    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"]
Jul 28 20:29:19.669331 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.669293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c17dde-1f28-4dd0-a3bc-41534d3e7749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.669538 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.669344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbcnd\" (UniqueName: \"kubernetes.io/projected/46c17dde-1f28-4dd0-a3bc-41534d3e7749-kube-api-access-vbcnd\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.770629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.770597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c17dde-1f28-4dd0-a3bc-41534d3e7749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.770794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.770633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbcnd\" (UniqueName: \"kubernetes.io/projected/46c17dde-1f28-4dd0-a3bc-41534d3e7749-kube-api-access-vbcnd\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.770991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.770972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46c17dde-1f28-4dd0-a3bc-41534d3e7749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.783304 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.783272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbcnd\" (UniqueName: \"kubernetes.io/projected/46c17dde-1f28-4dd0-a3bc-41534d3e7749-kube-api-access-vbcnd\") pod \"cert-manager-operator-controller-manager-55c444bf86-gnj7d\" (UID: \"46c17dde-1f28-4dd0-a3bc-41534d3e7749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:19.881460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:19.881378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"
Jul 28 20:29:20.024731 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:20.024694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d"]
Jul 28 20:29:20.030534 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:20.030497    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c17dde_1f28_4dd0_a3bc_41534d3e7749.slice/crio-3c363bdd73f2d71a58253fb9a0ed625b60fad6523902c0b73ade931a17266ff7 WatchSource:0}: Error finding container 3c363bdd73f2d71a58253fb9a0ed625b60fad6523902c0b73ade931a17266ff7: Status 404 returned error can't find the container with id 3c363bdd73f2d71a58253fb9a0ed625b60fad6523902c0b73ade931a17266ff7
Jul 28 20:29:20.473212 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:20.473176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d" event={"ID":"46c17dde-1f28-4dd0-a3bc-41534d3e7749","Type":"ContainerStarted","Data":"3c363bdd73f2d71a58253fb9a0ed625b60fad6523902c0b73ade931a17266ff7"}
Jul 28 20:29:21.224162 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:21.224124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:21.322664 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:21.322617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:22.482537 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:22.482447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d" event={"ID":"46c17dde-1f28-4dd0-a3bc-41534d3e7749","Type":"ContainerStarted","Data":"f0361ecfb0c98618d95ec34ad19d95aa6603cff67eee3594859ff762a99b9b2b"}
Jul 28 20:29:22.509984 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:22.509918    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-gnj7d" podStartSLOduration=1.5340357999999998 podStartE2EDuration="3.509897288s" totalImagesPullingTime="1.975861488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:22.506944149 +0000 UTC m=+257.692048854" watchObservedRunningTime="2026-07-28 20:29:22.509897288 +0000 UTC m=+257.695001996"
Jul 28 20:29:23.005181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.005148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.191798 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.191760    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.281946 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.281872    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"]
Jul 28 20:29:23.284114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.284098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.286629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.286608    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:29:23.287279 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.287257    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dgzk9\""
Jul 28 20:29:23.287609 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.287591    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:29:23.299862 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.299842    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"]
Jul 28 20:29:23.317403 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.317339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.402963 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.402922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.402963 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.402965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.403203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.402988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwr6d\" (UniqueName: \"kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.456502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.456463    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8vmw"]
Jul 28 20:29:23.458857 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.458832    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.461447 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.461427    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cgxsz\""
Jul 28 20:29:23.461817 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.461799    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 20:29:23.462041 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.462027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 20:29:23.476552 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.476527    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8vmw"]
Jul 28 20:29:23.503524 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.503494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.503991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.503573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.503991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.503621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwr6d\" (UniqueName: \"kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.503991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.503943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.503991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.503960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.524649 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.524616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwr6d\" (UniqueName: \"kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.593780 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.593694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:23.604155 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.604125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.604260 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.604202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6x5\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-kube-api-access-8k6x5\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.704857 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.704824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.705005 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.704871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6x5\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-kube-api-access-8k6x5\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.716906 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.716879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.716906 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.716905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6x5\" (UniqueName: \"kubernetes.io/projected/6d521b26-2477-4dbb-803b-b364134fdc7b-kube-api-access-8k6x5\") pod \"cert-manager-webhook-6486ffd997-h8vmw\" (UID: \"6d521b26-2477-4dbb-803b-b364134fdc7b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.728810 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.728780    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"]
Jul 28 20:29:23.732667 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:23.732638    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd791b63_7a68_4696_9eec_0c1437ce56b3.slice/crio-0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1 WatchSource:0}: Error finding container 0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1: Status 404 returned error can't find the container with id 0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1
Jul 28 20:29:23.777127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.777105    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:23.908436 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:23.908257    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-h8vmw"]
Jul 28 20:29:23.910982 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:23.910954    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d521b26_2477_4dbb_803b_b364134fdc7b.slice/crio-cc3305a2c1396f8e7dd6782748e9673356b1ba0ab85995bbf257e16ed6a554ea WatchSource:0}: Error finding container cc3305a2c1396f8e7dd6782748e9673356b1ba0ab85995bbf257e16ed6a554ea: Status 404 returned error can't find the container with id cc3305a2c1396f8e7dd6782748e9673356b1ba0ab85995bbf257e16ed6a554ea
Jul 28 20:29:24.491428 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:24.491394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw" event={"ID":"6d521b26-2477-4dbb-803b-b364134fdc7b","Type":"ContainerStarted","Data":"cc3305a2c1396f8e7dd6782748e9673356b1ba0ab85995bbf257e16ed6a554ea"}
Jul 28 20:29:24.492823 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:24.492799    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerID="dfeb0f3dd1bc13a8b955592ab2d16a7c97cfa299dd2b8e1409dab29735f9a945" exitCode=0
Jul 28 20:29:24.492939 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:24.492842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29" event={"ID":"fd791b63-7a68-4696-9eec-0c1437ce56b3","Type":"ContainerDied","Data":"dfeb0f3dd1bc13a8b955592ab2d16a7c97cfa299dd2b8e1409dab29735f9a945"}
Jul 28 20:29:24.492939 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:24.492873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29" event={"ID":"fd791b63-7a68-4696-9eec-0c1437ce56b3","Type":"ContainerStarted","Data":"0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1"}
Jul 28 20:29:25.553465 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:25.553422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:25.882708 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:25.882617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:26.115844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.115800    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"]
Jul 28 20:29:26.119997 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.119974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.122301 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.122271    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gcc4t\""
Jul 28 20:29:26.126624 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.126589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"]
Jul 28 20:29:26.230464 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.230351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vk2\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-kube-api-access-t7vk2\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.230464 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.230435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.331508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.331461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.331699 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.331560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vk2\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-kube-api-access-t7vk2\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.340202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.340175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.340345 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.340266    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vk2\" (UniqueName: \"kubernetes.io/projected/bc343057-e953-4d81-9c34-64c27fed38cc-kube-api-access-t7vk2\") pod \"cert-manager-cainjector-559cdd8c87-gvfvv\" (UID: \"bc343057-e953-4d81-9c34-64c27fed38cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:26.431938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:26.431900    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"
Jul 28 20:29:27.707509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:27.707485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv"]
Jul 28 20:29:27.709858 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:27.709831    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc343057_e953_4d81_9c34_64c27fed38cc.slice/crio-36e3fddb168693d3374424f51549b3936da9c3a36560598d2e49bc7d61a1c34e WatchSource:0}: Error finding container 36e3fddb168693d3374424f51549b3936da9c3a36560598d2e49bc7d61a1c34e: Status 404 returned error can't find the container with id 36e3fddb168693d3374424f51549b3936da9c3a36560598d2e49bc7d61a1c34e
Jul 28 20:29:28.013002 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.012963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:28.076847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.076811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:28.514268 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.514238    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerID="e21dea7e13a5136a892b4058dec72ccbef9181f4615badfafb5bb940b64e348c" exitCode=0
Jul 28 20:29:28.514462 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.514308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29" event={"ID":"fd791b63-7a68-4696-9eec-0c1437ce56b3","Type":"ContainerDied","Data":"e21dea7e13a5136a892b4058dec72ccbef9181f4615badfafb5bb940b64e348c"}
Jul 28 20:29:28.515923 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.515895    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv" event={"ID":"bc343057-e953-4d81-9c34-64c27fed38cc","Type":"ContainerStarted","Data":"fdc988559aedda09ec0d5f13cd7257b0703344e70e3abc875d7f47d9f307d3d6"}
Jul 28 20:29:28.516016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.515933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv" event={"ID":"bc343057-e953-4d81-9c34-64c27fed38cc","Type":"ContainerStarted","Data":"36e3fddb168693d3374424f51549b3936da9c3a36560598d2e49bc7d61a1c34e"}
Jul 28 20:29:28.517438 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.517416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw" event={"ID":"6d521b26-2477-4dbb-803b-b364134fdc7b","Type":"ContainerStarted","Data":"e8c15e30da3e0a4de11527eb966bfc71a86ab2a5f6379d9440a5208ac94283f8"}
Jul 28 20:29:28.517557 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.517539    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:28.545135 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.545021    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gvfvv" podStartSLOduration=2.545002695 podStartE2EDuration="2.545002695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:28.544508847 +0000 UTC m=+263.729613554" watchObservedRunningTime="2026-07-28 20:29:28.545002695 +0000 UTC m=+263.730107402"
Jul 28 20:29:28.560881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:28.560832    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw" podStartSLOduration=1.899803771 podStartE2EDuration="5.560812146s" totalImagesPullingTime="3.661008375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:28.559929267 +0000 UTC m=+263.745033974" watchObservedRunningTime="2026-07-28 20:29:28.560812146 +0000 UTC m=+263.745916853"
Jul 28 20:29:29.523453 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:29.523414    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerID="423a4c9ef8f7e45e220f2b359ee363729ad2ddf43ccff4dde7b4b2476bc1eced" exitCode=0
Jul 28 20:29:29.523888 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:29.523500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29" event={"ID":"fd791b63-7a68-4696-9eec-0c1437ce56b3","Type":"ContainerDied","Data":"423a4c9ef8f7e45e220f2b359ee363729ad2ddf43ccff4dde7b4b2476bc1eced"}
Jul 28 20:29:30.501386 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.501326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:30.652546 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.652514    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:30.771682 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.771595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwr6d\" (UniqueName: \"kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d\") pod \"fd791b63-7a68-4696-9eec-0c1437ce56b3\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") "
Jul 28 20:29:30.771682 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.771653    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util\") pod \"fd791b63-7a68-4696-9eec-0c1437ce56b3\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") "
Jul 28 20:29:30.771910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.771696    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle\") pod \"fd791b63-7a68-4696-9eec-0c1437ce56b3\" (UID: \"fd791b63-7a68-4696-9eec-0c1437ce56b3\") "
Jul 28 20:29:30.772067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.772045    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle" (OuterVolumeSpecName: "bundle") pod "fd791b63-7a68-4696-9eec-0c1437ce56b3" (UID: "fd791b63-7a68-4696-9eec-0c1437ce56b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:30.773816 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.773784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d" (OuterVolumeSpecName: "kube-api-access-zwr6d") pod "fd791b63-7a68-4696-9eec-0c1437ce56b3" (UID: "fd791b63-7a68-4696-9eec-0c1437ce56b3"). InnerVolumeSpecName "kube-api-access-zwr6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:30.775832 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.775810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util" (OuterVolumeSpecName: "util") pod "fd791b63-7a68-4696-9eec-0c1437ce56b3" (UID: "fd791b63-7a68-4696-9eec-0c1437ce56b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:30.872314 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.872283    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwr6d\" (UniqueName: \"kubernetes.io/projected/fd791b63-7a68-4696-9eec-0c1437ce56b3-kube-api-access-zwr6d\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:30.872314 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.872309    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:30.872314 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:30.872318    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd791b63-7a68-4696-9eec-0c1437ce56b3-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:31.533095 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:31.533065    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29"
Jul 28 20:29:31.533258 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:31.533065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxvr29" event={"ID":"fd791b63-7a68-4696-9eec-0c1437ce56b3","Type":"ContainerDied","Data":"0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1"}
Jul 28 20:29:31.533258 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:31.533188    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cc39a55854772b734cef657cf75284fef2068d97b426f2ee15f3e7c31c669b1"
Jul 28 20:29:32.872838 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:32.872807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:34.526175 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:34.526138    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-h8vmw"
Jul 28 20:29:38.061797 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:38.061755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:38.208163 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:38.208120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:39.749212 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749178    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"]
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749453    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="util"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749464    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="util"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749476    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="extract"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749482    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="extract"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749491    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="pull"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749496    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="pull"
Jul 28 20:29:39.749716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.749540    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd791b63-7a68-4696-9eec-0c1437ce56b3" containerName="extract"
Jul 28 20:29:39.785604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.785571    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"]
Jul 28 20:29:39.785756 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.785658    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.788034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.788006    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:29:39.788162 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.788012    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 20:29:39.788162 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.788032    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wz54s\""
Jul 28 20:29:39.842628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.842590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1fae4c6-ff2b-4186-b606-bd891863cdb9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.842628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.842629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8qn5\" (UniqueName: \"kubernetes.io/projected/a1fae4c6-ff2b-4186-b606-bd891863cdb9-kube-api-access-n8qn5\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.943951 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.943915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1fae4c6-ff2b-4186-b606-bd891863cdb9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.943951 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.943957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8qn5\" (UniqueName: \"kubernetes.io/projected/a1fae4c6-ff2b-4186-b606-bd891863cdb9-kube-api-access-n8qn5\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.944388 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.944331    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1fae4c6-ff2b-4186-b606-bd891863cdb9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:39.952319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:39.952295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8qn5\" (UniqueName: \"kubernetes.io/projected/a1fae4c6-ff2b-4186-b606-bd891863cdb9-kube-api-access-n8qn5\") pod \"openshift-lws-operator-fd8ccff4c-nwkxc\" (UID: \"a1fae4c6-ff2b-4186-b606-bd891863cdb9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:40.095296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:40.095202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"
Jul 28 20:29:40.263259 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:40.263226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc"]
Jul 28 20:29:40.266445 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:40.266417    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1fae4c6_ff2b_4186_b606_bd891863cdb9.slice/crio-9951aac7ce50121230e71eee175827b786b5487abb86d952d291a63596a9717e WatchSource:0}: Error finding container 9951aac7ce50121230e71eee175827b786b5487abb86d952d291a63596a9717e: Status 404 returned error can't find the container with id 9951aac7ce50121230e71eee175827b786b5487abb86d952d291a63596a9717e
Jul 28 20:29:40.572049 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:40.571998    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc" event={"ID":"a1fae4c6-ff2b-4186-b606-bd891863cdb9","Type":"ContainerStarted","Data":"9951aac7ce50121230e71eee175827b786b5487abb86d952d291a63596a9717e"}
Jul 28 20:29:47.601847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:47.601810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc" event={"ID":"a1fae4c6-ff2b-4186-b606-bd891863cdb9","Type":"ContainerStarted","Data":"02d5def0f12ed9b708421992ebd36c20bdc5786777b3f3dc0247c528bd26c260"}
Jul 28 20:29:47.619347 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:47.619283    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nwkxc" podStartSLOduration=1.811067393 podStartE2EDuration="8.619263104s" totalImagesPullingTime="6.808195711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:47.617093413 +0000 UTC m=+282.802198120" watchObservedRunningTime="2026-07-28 20:29:47.619263104 +0000 UTC m=+282.804367812"
Jul 28 20:29:48.373179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:48.373150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:49.851866 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.851816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:49.960589 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.960555    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"]
Jul 28 20:29:49.962844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.962827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:49.965869 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.965848    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 20:29:49.966159 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.966143    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 20:29:49.966817 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.966800    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dgzk9\""
Jul 28 20:29:49.975895 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:49.975868    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"]
Jul 28 20:29:50.025222 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.025190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.025406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.025230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9b52\" (UniqueName: \"kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.025406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.025321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.101378 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.101336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:50.126738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.126671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.126738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.126718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.126918 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.126745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9b52\" (UniqueName: \"kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.127112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.127090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.127193 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.127143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.136375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.136333    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9b52\" (UniqueName: \"kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.272030 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.271998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:50.403829 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.403746    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"]
Jul 28 20:29:50.406782 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:29:50.406750    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65514335_79b2_407c_a04d_c693a2ccfc02.slice/crio-99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219 WatchSource:0}: Error finding container 99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219: Status 404 returned error can't find the container with id 99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219
Jul 28 20:29:50.616759 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.616724    2576 generic.go:356] "Generic (PLEG): container finished" podID="65514335-79b2-407c-a04d-c693a2ccfc02" containerID="bd6723f63a0d1c0ed95db5d2163cce42eced4a55d810326b8bc8c59e4fafe17b" exitCode=0
Jul 28 20:29:50.616937 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.616813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x" event={"ID":"65514335-79b2-407c-a04d-c693a2ccfc02","Type":"ContainerDied","Data":"bd6723f63a0d1c0ed95db5d2163cce42eced4a55d810326b8bc8c59e4fafe17b"}
Jul 28 20:29:50.616937 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:50.616863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x" event={"ID":"65514335-79b2-407c-a04d-c693a2ccfc02","Type":"ContainerStarted","Data":"99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219"}
Jul 28 20:29:51.623337 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:51.623295    2576 generic.go:356] "Generic (PLEG): container finished" podID="65514335-79b2-407c-a04d-c693a2ccfc02" containerID="7a35a6c4baaaa1dd104675aa8d1f6de73f5fd946abed3d71d627f79c532878bb" exitCode=0
Jul 28 20:29:51.623727 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:51.623346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x" event={"ID":"65514335-79b2-407c-a04d-c693a2ccfc02","Type":"ContainerDied","Data":"7a35a6c4baaaa1dd104675aa8d1f6de73f5fd946abed3d71d627f79c532878bb"}
Jul 28 20:29:52.630037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:52.629998    2576 generic.go:356] "Generic (PLEG): container finished" podID="65514335-79b2-407c-a04d-c693a2ccfc02" containerID="9651440cb051f45b11353ffe2e6f27c7361041e97136a9c2e6c09fcc6ce08759" exitCode=0
Jul 28 20:29:52.630430 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:52.630079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x" event={"ID":"65514335-79b2-407c-a04d-c693a2ccfc02","Type":"ContainerDied","Data":"9651440cb051f45b11353ffe2e6f27c7361041e97136a9c2e6c09fcc6ce08759"}
Jul 28 20:29:53.755396 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.755355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:53.856289 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.856254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util\") pod \"65514335-79b2-407c-a04d-c693a2ccfc02\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") "
Jul 28 20:29:53.856289 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.856294    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle\") pod \"65514335-79b2-407c-a04d-c693a2ccfc02\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") "
Jul 28 20:29:53.856543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.856334    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9b52\" (UniqueName: \"kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52\") pod \"65514335-79b2-407c-a04d-c693a2ccfc02\" (UID: \"65514335-79b2-407c-a04d-c693a2ccfc02\") "
Jul 28 20:29:53.857054 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.857026    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle" (OuterVolumeSpecName: "bundle") pod "65514335-79b2-407c-a04d-c693a2ccfc02" (UID: "65514335-79b2-407c-a04d-c693a2ccfc02"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:53.858463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.858436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52" (OuterVolumeSpecName: "kube-api-access-z9b52") pod "65514335-79b2-407c-a04d-c693a2ccfc02" (UID: "65514335-79b2-407c-a04d-c693a2ccfc02"). InnerVolumeSpecName "kube-api-access-z9b52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:53.861998 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.861974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util" (OuterVolumeSpecName: "util") pod "65514335-79b2-407c-a04d-c693a2ccfc02" (UID: "65514335-79b2-407c-a04d-c693a2ccfc02"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:29:53.957019 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.956930    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:53.957019 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.956961    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65514335-79b2-407c-a04d-c693a2ccfc02-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:53.957019 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:53.956974    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9b52\" (UniqueName: \"kubernetes.io/projected/65514335-79b2-407c-a04d-c693a2ccfc02-kube-api-access-z9b52\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:29:54.023150 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:54.023117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:54.084835 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:54.084793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:54.640042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:54.640016    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x"
Jul 28 20:29:54.640215 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:54.640014    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgpr4x" event={"ID":"65514335-79b2-407c-a04d-c693a2ccfc02","Type":"ContainerDied","Data":"99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219"}
Jul 28 20:29:54.640215 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:29:54.640142    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99d3a24d95af879371cdfeb9b652042959b176f12abe59020c8d3a1a00e1c219"
Jul 28 20:30:05.164134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:05.164094    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 20:30:06.079484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:06.079447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:06.594149 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:06.594109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:06.849698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:06.849608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.134279 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.134193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.251796 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.251764    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.316906 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.316869    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"]
Jul 28 20:30:07.317139 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317127    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="util"
Jul 28 20:30:07.317181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317140    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="util"
Jul 28 20:30:07.317181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317154    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="pull"
Jul 28 20:30:07.317181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317159    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="pull"
Jul 28 20:30:07.317181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317169    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="extract"
Jul 28 20:30:07.317181 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317174    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="extract"
Jul 28 20:30:07.317328 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.317218    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="65514335-79b2-407c-a04d-c693a2ccfc02" containerName="extract"
Jul 28 20:30:07.319099 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.319083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.324288 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.324260    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 20:30:07.324463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.324266    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 20:30:07.326327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.326303    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 20:30:07.326460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.326303    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sr7hb\""
Jul 28 20:30:07.326460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.326380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 20:30:07.338726 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.338694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"]
Jul 28 20:30:07.377076 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.377036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.377298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.377125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-manifests\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.377298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.377184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-tmp\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.377298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.377263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbxl6\" (UniqueName: \"kubernetes.io/projected/cad0a907-5f5c-4903-8987-d7b278f7c43d-kube-api-access-tbxl6\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.377464 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.377324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.478676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.478581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbxl6\" (UniqueName: \"kubernetes.io/projected/cad0a907-5f5c-4903-8987-d7b278f7c43d-kube-api-access-tbxl6\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.478676 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.478633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.478920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.478726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.478920 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.478852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-manifests\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.479023 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.478944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-tmp\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.479232 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.479205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-manifests\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.481082 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.481061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cad0a907-5f5c-4903-8987-d7b278f7c43d-tmp\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.481173 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.481127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.481173 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.481152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cad0a907-5f5c-4903-8987-d7b278f7c43d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.488926 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.488895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbxl6\" (UniqueName: \"kubernetes.io/projected/cad0a907-5f5c-4903-8987-d7b278f7c43d-kube-api-access-tbxl6\") pod \"opendatahub-operator-controller-manager-56794f8cc8-dgcqq\" (UID: \"cad0a907-5f5c-4903-8987-d7b278f7c43d\") " pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.595965 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.595900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.630275 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.630238    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:07.697417 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.697375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.789816 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.789792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"]
Jul 28 20:30:07.792096 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:07.792055    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad0a907_5f5c_4903_8987_d7b278f7c43d.slice/crio-ada9787178d45c6b451034f16d5bc478187725958207423c99dd65c876e63e0b WatchSource:0}: Error finding container ada9787178d45c6b451034f16d5bc478187725958207423c99dd65c876e63e0b: Status 404 returned error can't find the container with id ada9787178d45c6b451034f16d5bc478187725958207423c99dd65c876e63e0b
Jul 28 20:30:07.793153 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.793125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.794063 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.794048    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:30:07.870619 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.870573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:07.971582 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:07.971544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:08.070552 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:08.070455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:08.700335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:08.700301    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq" event={"ID":"cad0a907-5f5c-4903-8987-d7b278f7c43d","Type":"ContainerStarted","Data":"ada9787178d45c6b451034f16d5bc478187725958207423c99dd65c876e63e0b"}
Jul 28 20:30:11.716269 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:11.716236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq" event={"ID":"cad0a907-5f5c-4903-8987-d7b278f7c43d","Type":"ContainerStarted","Data":"9fd1f3b7ba4cb92dd2a8129f3d74dc7082a6d7e77c580644da9b6a222c030ece"}
Jul 28 20:30:12.388587 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:12.388502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:12.485736 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:12.485697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:12.722044 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:12.721957    2576 generic.go:356] "Generic (PLEG): container finished" podID="cad0a907-5f5c-4903-8987-d7b278f7c43d" containerID="9fd1f3b7ba4cb92dd2a8129f3d74dc7082a6d7e77c580644da9b6a222c030ece" exitCode=0
Jul 28 20:30:12.722428 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:12.722045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq" event={"ID":"cad0a907-5f5c-4903-8987-d7b278f7c43d","Type":"ContainerDied","Data":"9fd1f3b7ba4cb92dd2a8129f3d74dc7082a6d7e77c580644da9b6a222c030ece"}
Jul 28 20:30:13.730517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:13.730475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq" event={"ID":"cad0a907-5f5c-4903-8987-d7b278f7c43d","Type":"ContainerStarted","Data":"98c1649406a655b0a38131c7c867f7a543708f24a8c2dbeb470fc697f07bdb8f"}
Jul 28 20:30:13.731021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:13.730588    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:15.236096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:15.236061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:15.327584 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:15.327540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:24.736930 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:24.736895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq"
Jul 28 20:30:24.772019 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:24.771960    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56794f8cc8-dgcqq" podStartSLOduration=13.269398777 podStartE2EDuration="17.77193747s" totalImagesPullingTime="3.464042043s" totalInitContainerRuntime="1.03849665s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:13.764858239 +0000 UTC m=+308.949962947" watchObservedRunningTime="2026-07-28 20:30:24.77193747 +0000 UTC m=+319.957042179"
Jul 28 20:30:26.728478 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:26.728439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:26.979334 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:26.979247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:27.124585 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.124552    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"]
Jul 28 20:30:27.127091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.127076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.130038 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.130015    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 20:30:27.130154 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.130016    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 20:30:27.130374 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.130345    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 20:30:27.130974 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.130958    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 20:30:27.156079 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.156054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"]
Jul 28 20:30:27.247470 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.247443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tmp\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.247634 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.247476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9njf\" (UniqueName: \"kubernetes.io/projected/32ec4a8d-5e06-4818-b37f-0fcda891eb77-kube-api-access-t9njf\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.247634 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.247574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.253745 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.253705    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:27.324066 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.324027    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-84b9p"]
Jul 28 20:30:27.326341 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.326322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.329756 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.329738    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 20:30:27.333881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.333865    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7lnf7\""
Jul 28 20:30:27.348327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.348293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.348513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.348336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tmp\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.348513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.348355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9njf\" (UniqueName: \"kubernetes.io/projected/32ec4a8d-5e06-4818-b37f-0fcda891eb77-kube-api-access-t9njf\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.348513 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:27.348458    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 20:30:27.348680 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:27.348524    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs podName:32ec4a8d-5e06-4818-b37f-0fcda891eb77 nodeName:}" failed. No retries permitted until 2026-07-28 20:30:27.848503247 +0000 UTC m=+323.033607931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs") pod "kube-auth-proxy-7d6cc5795b-cxh4l" (UID: "32ec4a8d-5e06-4818-b37f-0fcda891eb77") : secret "kube-auth-proxy-tls" not found
Jul 28 20:30:27.350639 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.350620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tmp\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.364484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.364453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-84b9p"]
Jul 28 20:30:27.376725 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.376698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9njf\" (UniqueName: \"kubernetes.io/projected/32ec4a8d-5e06-4818-b37f-0fcda891eb77-kube-api-access-t9njf\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.449227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.449191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.449442 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.449291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsv8d\" (UniqueName: \"kubernetes.io/projected/bbdc44c6-c6ff-4661-8017-96def9de0514-kube-api-access-qsv8d\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.459696 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.459655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:27.549962 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.549880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsv8d\" (UniqueName: \"kubernetes.io/projected/bbdc44c6-c6ff-4661-8017-96def9de0514-kube-api-access-qsv8d\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.550112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.549966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.550112 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:27.550060    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 20:30:27.550199 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:27.550117    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert podName:bbdc44c6-c6ff-4661-8017-96def9de0514 nodeName:}" failed. No retries permitted until 2026-07-28 20:30:28.050101796 +0000 UTC m=+323.235206494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert") pod "odh-model-controller-7998c67dc4-84b9p" (UID: "bbdc44c6-c6ff-4661-8017-96def9de0514") : secret "odh-model-controller-webhook-cert" not found
Jul 28 20:30:27.569321 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.569286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsv8d\" (UniqueName: \"kubernetes.io/projected/bbdc44c6-c6ff-4661-8017-96def9de0514-kube-api-access-qsv8d\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:27.792601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.792568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:27.852633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.852546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:27.855379 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:27.855340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32ec4a8d-5e06-4818-b37f-0fcda891eb77-tls-certs\") pod \"kube-auth-proxy-7d6cc5795b-cxh4l\" (UID: \"32ec4a8d-5e06-4818-b37f-0fcda891eb77\") " pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:28.003864 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.003827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:28.035667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.035633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"
Jul 28 20:30:28.054319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.054293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:28.056686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.056660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbdc44c6-c6ff-4661-8017-96def9de0514-cert\") pod \"odh-model-controller-7998c67dc4-84b9p\" (UID: \"bbdc44c6-c6ff-4661-8017-96def9de0514\") " pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:28.235772 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.235735    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:28.239502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.239474    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l"]
Jul 28 20:30:28.246889 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:28.246858    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ec4a8d_5e06_4818_b37f_0fcda891eb77.slice/crio-693f3b95eaf062fa6d236005800deff14424a818193f06d4669a0bee49446eac WatchSource:0}: Error finding container 693f3b95eaf062fa6d236005800deff14424a818193f06d4669a0bee49446eac: Status 404 returned error can't find the container with id 693f3b95eaf062fa6d236005800deff14424a818193f06d4669a0bee49446eac
Jul 28 20:30:28.272702 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.272664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:28.386701 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.386525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-84b9p"]
Jul 28 20:30:28.389227 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:28.389199    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdc44c6_c6ff_4661_8017_96def9de0514.slice/crio-7a4dab8ee24439a37dfd30494682ae1aa81bb8c89e57ce82b9fedae9f5ee9060 WatchSource:0}: Error finding container 7a4dab8ee24439a37dfd30494682ae1aa81bb8c89e57ce82b9fedae9f5ee9060: Status 404 returned error can't find the container with id 7a4dab8ee24439a37dfd30494682ae1aa81bb8c89e57ce82b9fedae9f5ee9060
Jul 28 20:30:28.740191 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.740152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:28.794623 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.794585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p" event={"ID":"bbdc44c6-c6ff-4661-8017-96def9de0514","Type":"ContainerStarted","Data":"7a4dab8ee24439a37dfd30494682ae1aa81bb8c89e57ce82b9fedae9f5ee9060"}
Jul 28 20:30:28.795983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.795951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l" event={"ID":"32ec4a8d-5e06-4818-b37f-0fcda891eb77","Type":"ContainerStarted","Data":"693f3b95eaf062fa6d236005800deff14424a818193f06d4669a0bee49446eac"}
Jul 28 20:30:28.901202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.901162    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9996g"]
Jul 28 20:30:28.903896 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.903881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:28.910244 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.910208    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7rt6m\""
Jul 28 20:30:28.911127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.911101    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 20:30:28.926882 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:28.926850    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9996g"]
Jul 28 20:30:29.063526 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.063380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488a6ede-7923-4a5d-911c-11425d3e2ced-tls-certs\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.063526 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.063459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gxv\" (UniqueName: \"kubernetes.io/projected/488a6ede-7923-4a5d-911c-11425d3e2ced-kube-api-access-x7gxv\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.139122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.139082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:29.164774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.164735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488a6ede-7923-4a5d-911c-11425d3e2ced-tls-certs\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.164948 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.164792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gxv\" (UniqueName: \"kubernetes.io/projected/488a6ede-7923-4a5d-911c-11425d3e2ced-kube-api-access-x7gxv\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.167141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.167110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488a6ede-7923-4a5d-911c-11425d3e2ced-tls-certs\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.190425 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.190390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gxv\" (UniqueName: \"kubernetes.io/projected/488a6ede-7923-4a5d-911c-11425d3e2ced-kube-api-access-x7gxv\") pod \"model-serving-api-54d89bd5fc-9996g\" (UID: \"488a6ede-7923-4a5d-911c-11425d3e2ced\") " pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.214042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.214003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:29.381662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.381624    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9996g"]
Jul 28 20:30:29.384846 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:29.384692    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488a6ede_7923_4a5d_911c_11425d3e2ced.slice/crio-f5c5de4db05449d969197eba2388c50fe7e1752022b11f12f0f0fbfd9005ff43 WatchSource:0}: Error finding container f5c5de4db05449d969197eba2388c50fe7e1752022b11f12f0f0fbfd9005ff43: Status 404 returned error can't find the container with id f5c5de4db05449d969197eba2388c50fe7e1752022b11f12f0f0fbfd9005ff43
Jul 28 20:30:29.718771 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.718682    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:29.721648 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.721626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.725410 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.725354    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 20:30:29.725410 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.725400    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 20:30:29.725615 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.725449    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dsphz\""
Jul 28 20:30:29.725615 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.725455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 20:30:29.725938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.725920    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 20:30:29.743201 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.743179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:29.758080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.758054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:29.802600 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.802563    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9996g" event={"ID":"488a6ede-7923-4a5d-911c-11425d3e2ced","Type":"ContainerStarted","Data":"f5c5de4db05449d969197eba2388c50fe7e1752022b11f12f0f0fbfd9005ff43"}
Jul 28 20:30:29.838928 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.838893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:29.870929 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.870898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871103 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.870936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42hv8\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871103 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.870978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871103 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.871059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.871118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.871175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.871243 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.871206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.971642 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.971642 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.971861 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.971861 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.971861 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42hv8\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.972018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.972018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.971933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.972646 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.972590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.974485 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.974442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.974642 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.974608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.974711 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.974672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.974765 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.974741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.983640 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.983553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42hv8\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:29.983840 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:29.983707    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4qjk7\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:30.031581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:30.031548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:30.194762 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:30.194720    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:30.200540 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:30.200503    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf159c3bb_1bd9_42da_a951_a3a441f03c38.slice/crio-ff113f9f1e6d5dcd777211b37778b9379084e986b2ecdc9b39a4f8d16fbce29b WatchSource:0}: Error finding container ff113f9f1e6d5dcd777211b37778b9379084e986b2ecdc9b39a4f8d16fbce29b: Status 404 returned error can't find the container with id ff113f9f1e6d5dcd777211b37778b9379084e986b2ecdc9b39a4f8d16fbce29b
Jul 28 20:30:30.810137 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:30.810091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" event={"ID":"f159c3bb-1bd9-42da-a951-a3a441f03c38","Type":"ContainerStarted","Data":"ff113f9f1e6d5dcd777211b37778b9379084e986b2ecdc9b39a4f8d16fbce29b"}
Jul 28 20:30:31.018659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:31.018472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:34.379673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.379632    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 20:30:34.379965 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.379715    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 20:30:34.831522 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.831481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9996g" event={"ID":"488a6ede-7923-4a5d-911c-11425d3e2ced","Type":"ContainerStarted","Data":"4df6692c58ca7d99c067367462dc4d019ba4bcf6a15faa6084c9a138ee15b1b4"}
Jul 28 20:30:34.831719 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.831612    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:34.833098 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.833074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" event={"ID":"f159c3bb-1bd9-42da-a951-a3a441f03c38","Type":"ContainerStarted","Data":"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"}
Jul 28 20:30:34.833211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.833142    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:34.834528 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.834508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p" event={"ID":"bbdc44c6-c6ff-4661-8017-96def9de0514","Type":"ContainerStarted","Data":"a76caab4fd6df9b65e2e1099cc4bd122d6a15d963488830097f0e75df712bcf4"}
Jul 28 20:30:34.834628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.834561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:34.836185 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.836165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l" event={"ID":"32ec4a8d-5e06-4818-b37f-0fcda891eb77","Type":"ContainerStarted","Data":"cd6d597a94b900c5c17f7d25926c615c20c014f0c610f6796e90e7ea6b7d07ab"}
Jul 28 20:30:34.876171 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.876116    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9996g" podStartSLOduration=2.779647495 podStartE2EDuration="6.876103221s" totalImagesPullingTime="4.096455726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:34.875278353 +0000 UTC m=+330.060383060" watchObservedRunningTime="2026-07-28 20:30:34.876103221 +0000 UTC m=+330.061207905"
Jul 28 20:30:34.981047 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.980991    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d6cc5795b-cxh4l" podStartSLOduration=1.889631675 podStartE2EDuration="7.980974465s" totalImagesPullingTime="6.09134279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:34.912516762 +0000 UTC m=+330.097621467" watchObservedRunningTime="2026-07-28 20:30:34.980974465 +0000 UTC m=+330.166079170"
Jul 28 20:30:34.981728 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:34.981699    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" podStartSLOduration=1.805454042 podStartE2EDuration="5.981686973s" totalImagesPullingTime="4.176232931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:34.980380657 +0000 UTC m=+330.165485356" watchObservedRunningTime="2026-07-28 20:30:34.981686973 +0000 UTC m=+330.166791678"
Jul 28 20:30:35.025782 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.025749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:35.035584 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.035537    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p" podStartSLOduration=2.051153893 podStartE2EDuration="8.035522609s" totalImagesPullingTime="5.984368716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:35.03225066 +0000 UTC m=+330.217355367" watchObservedRunningTime="2026-07-28 20:30:35.035522609 +0000 UTC m=+330.220627315"
Jul 28 20:30:35.150334 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.150221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:35.697054 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.697017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:35.842697 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.842638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:35.983921 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:35.983832    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:36.125644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:36.125590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:38.648075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.648042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:38.729078 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.729044    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"]
Jul 28 20:30:38.731720 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.731701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.735934 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.735913    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 20:30:38.735934 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.735925    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-95866\""
Jul 28 20:30:38.745822 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.745800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7748fbf0-5dc4-4586-8191-91e14af8ab11-cert\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.745916 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.745866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcjhm\" (UniqueName: \"kubernetes.io/projected/7748fbf0-5dc4-4586-8191-91e14af8ab11-kube-api-access-kcjhm\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.772624 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.772582    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"]
Jul 28 20:30:38.799641 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.799611    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"]
Jul 28 20:30:38.806778 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.806747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:38.812116 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.812089    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 20:30:38.812412 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.812394    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5tmrp\""
Jul 28 20:30:38.823398 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.819952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"]
Jul 28 20:30:38.846670 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.846630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcjhm\" (UniqueName: \"kubernetes.io/projected/7748fbf0-5dc4-4586-8191-91e14af8ab11-kube-api-access-kcjhm\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.846853 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.846722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7748fbf0-5dc4-4586-8191-91e14af8ab11-cert\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.846853 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.846761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:38.846853 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.846827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5j8\" (UniqueName: \"kubernetes.io/projected/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-kube-api-access-bz5j8\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:38.848205 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.848175    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:38.848481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.848452    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" podUID="f159c3bb-1bd9-42da-a951-a3a441f03c38" containerName="discovery" containerID="cri-o://9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4" gracePeriod=30
Jul 28 20:30:38.849491 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.849470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7748fbf0-5dc4-4586-8191-91e14af8ab11-cert\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.879686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.879657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcjhm\" (UniqueName: \"kubernetes.io/projected/7748fbf0-5dc4-4586-8191-91e14af8ab11-kube-api-access-kcjhm\") pod \"kserve-controller-manager-d6f4b65b8-4tcks\" (UID: \"7748fbf0-5dc4-4586-8191-91e14af8ab11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:38.883410 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.883355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:38.947397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.947290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:38.947397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.947351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5j8\" (UniqueName: \"kubernetes.io/projected/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-kube-api-access-bz5j8\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:38.947628 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:38.947450    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 20:30:38.947628 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:38.947517    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert podName:8cefa4ed-509f-4ee8-97c2-aa0b01b290fd nodeName:}" failed. No retries permitted until 2026-07-28 20:30:39.447499336 +0000 UTC m=+334.632604019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert") pod "llmisvc-controller-manager-777c6bd7d5-h2gvl" (UID: "8cefa4ed-509f-4ee8-97c2-aa0b01b290fd") : secret "llmisvc-webhook-server-cert" not found
Jul 28 20:30:38.963742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:38.963712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5j8\" (UniqueName: \"kubernetes.io/projected/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-kube-api-access-bz5j8\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:39.015247 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.015198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:39.041833 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.041805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:39.091629 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.091600    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:39.148766 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.148944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.148944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148824    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42hv8\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.148944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.148944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148940    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.149154 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.148966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.149154 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.149013    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig\") pod \"f159c3bb-1bd9-42da-a951-a3a441f03c38\" (UID: \"f159c3bb-1bd9-42da-a951-a3a441f03c38\") "
Jul 28 20:30:39.149604 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.149540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:30:39.151994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.151958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token" (OuterVolumeSpecName: "istio-token") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:39.153881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.153854    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:30:39.153982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.153961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs" (OuterVolumeSpecName: "local-certs") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:30:39.154072 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.154055    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts" (OuterVolumeSpecName: "cacerts") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:30:39.154261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.154229    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:30:39.154517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.154492    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8" (OuterVolumeSpecName: "kube-api-access-42hv8") pod "f159c3bb-1bd9-42da-a951-a3a441f03c38" (UID: "f159c3bb-1bd9-42da-a951-a3a441f03c38"). InnerVolumeSpecName "kube-api-access-42hv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:30:39.195583 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.195543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:39.222007 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.221982    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"]
Jul 28 20:30:39.224062 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:39.224036    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7748fbf0_5dc4_4586_8191_91e14af8ab11.slice/crio-dc6cd6630656bf235269fbc26843d543dc6fb5e76bea4bb61557c274c17d79a2 WatchSource:0}: Error finding container dc6cd6630656bf235269fbc26843d543dc6fb5e76bea4bb61557c274c17d79a2: Status 404 returned error can't find the container with id dc6cd6630656bf235269fbc26843d543dc6fb5e76bea4bb61557c274c17d79a2
Jul 28 20:30:39.249804 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249776    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-token\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249804 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249802    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42hv8\" (UniqueName: \"kubernetes.io/projected/f159c3bb-1bd9-42da-a951-a3a441f03c38-kube-api-access-42hv8\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249815    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-ca-configmap\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249827    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-csr-dns-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249836    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f159c3bb-1bd9-42da-a951-a3a441f03c38-local-certs\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249845    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-istio-kubeconfig\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.249953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.249854    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f159c3bb-1bd9-42da-a951-a3a441f03c38-cacerts\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:30:39.388844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.388811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:39.452009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.451974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:39.454333 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.454311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cefa4ed-509f-4ee8-97c2-aa0b01b290fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h2gvl\" (UID: \"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:39.716932 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.716891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:39.852143 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.852117    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"]
Jul 28 20:30:39.853850 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:30:39.853819    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8cefa4ed_509f_4ee8_97c2_aa0b01b290fd.slice/crio-a3f4fc8ecec18bc8611dd1fc35dec85e34f529f4bc5a2d5373fa3dbf95085793 WatchSource:0}: Error finding container a3f4fc8ecec18bc8611dd1fc35dec85e34f529f4bc5a2d5373fa3dbf95085793: Status 404 returned error can't find the container with id a3f4fc8ecec18bc8611dd1fc35dec85e34f529f4bc5a2d5373fa3dbf95085793
Jul 28 20:30:39.862067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.862038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl" event={"ID":"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd","Type":"ContainerStarted","Data":"a3f4fc8ecec18bc8611dd1fc35dec85e34f529f4bc5a2d5373fa3dbf95085793"}
Jul 28 20:30:39.863396 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.863358    2576 generic.go:356] "Generic (PLEG): container finished" podID="f159c3bb-1bd9-42da-a951-a3a441f03c38" containerID="9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4" exitCode=0
Jul 28 20:30:39.863499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.863405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" event={"ID":"f159c3bb-1bd9-42da-a951-a3a441f03c38","Type":"ContainerDied","Data":"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"}
Jul 28 20:30:39.863499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.863434    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"
Jul 28 20:30:39.863499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.863454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7" event={"ID":"f159c3bb-1bd9-42da-a951-a3a441f03c38","Type":"ContainerDied","Data":"ff113f9f1e6d5dcd777211b37778b9379084e986b2ecdc9b39a4f8d16fbce29b"}
Jul 28 20:30:39.863499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.863486    2576 scope.go:117] "RemoveContainer" containerID="9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"
Jul 28 20:30:39.864865 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.864842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks" event={"ID":"7748fbf0-5dc4-4586-8191-91e14af8ab11","Type":"ContainerStarted","Data":"dc6cd6630656bf235269fbc26843d543dc6fb5e76bea4bb61557c274c17d79a2"}
Jul 28 20:30:39.877882 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.877864    2576 scope.go:117] "RemoveContainer" containerID="9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"
Jul 28 20:30:39.878168 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:30:39.878153    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4\": container with ID starting with 9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4 not found: ID does not exist" containerID="9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"
Jul 28 20:30:39.878219 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.878177    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4"} err="failed to get container status \"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4\": rpc error: code = NotFound desc = could not find container \"9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4\": container with ID starting with 9181250d6d4e73725de4c05ccac1c774e42a74b600638e5d5978f3d4e743b1c4 not found: ID does not exist"
Jul 28 20:30:39.891593 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.891553    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:39.901465 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:39.901441    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4qjk7"]
Jul 28 20:30:40.916548 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:40.916490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:41.063740 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:41.063698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:41.299780 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:41.299750    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f159c3bb-1bd9-42da-a951-a3a441f03c38" path="/var/lib/kubelet/pods/f159c3bb-1bd9-42da-a951-a3a441f03c38/volumes"
Jul 28 20:30:42.884744 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:42.884709    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl" event={"ID":"8cefa4ed-509f-4ee8-97c2-aa0b01b290fd","Type":"ContainerStarted","Data":"031d7dc8023ed5cee69f56eb7206ec75ca9f1a35626fcc115ecfb9482e136923"}
Jul 28 20:30:42.885395 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:42.884840    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:30:42.905500 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:42.905381    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl" podStartSLOduration=2.104838574 podStartE2EDuration="4.905345053s" totalImagesPullingTime="2.800506479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:42.903814674 +0000 UTC m=+338.088919384" watchObservedRunningTime="2026-07-28 20:30:42.905345053 +0000 UTC m=+338.090449760"
Jul 28 20:30:43.890777 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:43.890740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks" event={"ID":"7748fbf0-5dc4-4586-8191-91e14af8ab11","Type":"ContainerStarted","Data":"d2ba99b6d29f0ee566cb4b96eb9b916c1addf0c6c542a0cc8eeb3e73213addc4"}
Jul 28 20:30:43.891188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:43.890830    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:30:43.910250 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:43.910186    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks" podStartSLOduration=1.77667043 podStartE2EDuration="5.910171052s" totalImagesPullingTime="4.133500622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:30:43.907990145 +0000 UTC m=+339.093094862" watchObservedRunningTime="2026-07-28 20:30:43.910171052 +0000 UTC m=+339.095275757"
Jul 28 20:30:44.256481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:44.256441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:44.347198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:44.347164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:45.843791 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:45.843762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-84b9p"
Jul 28 20:30:45.845730 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:45.845710    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9996g"
Jul 28 20:30:47.868812 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:47.868773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:47.973851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:47.973810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:57.733227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:57.733148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:57.843665 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:30:57.843615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:13.893517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:13.893477    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h2gvl"
Jul 28 20:31:14.900905 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:14.900876    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4tcks"
Jul 28 20:31:28.136801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.136757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:28.232937 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.232896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:28.256167 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.256133    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:28.256636 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.256612    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f159c3bb-1bd9-42da-a951-a3a441f03c38" containerName="discovery"
Jul 28 20:31:28.256636 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.256635    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f159c3bb-1bd9-42da-a951-a3a441f03c38" containerName="discovery"
Jul 28 20:31:28.256827 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.256732    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f159c3bb-1bd9-42da-a951-a3a441f03c38" containerName="discovery"
Jul 28 20:31:28.259710 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.259692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:28.262764 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.262743    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:31:28.263613 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.263595    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:31:28.263694 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.263601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6jhpw\""
Jul 28 20:31:28.275296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.275274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:28.281821 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.281801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4dcp\" (UniqueName: \"kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp\") pod \"kuadrant-operator-catalog-5tgcm\" (UID: \"b462075b-8a03-419f-8cc9-38de96c8f347\") " pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:28.379429 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.379391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:28.383189 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.383170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4dcp\" (UniqueName: \"kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp\") pod \"kuadrant-operator-catalog-5tgcm\" (UID: \"b462075b-8a03-419f-8cc9-38de96c8f347\") " pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:28.391617 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.391563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4dcp\" (UniqueName: \"kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp\") pod \"kuadrant-operator-catalog-5tgcm\" (UID: \"b462075b-8a03-419f-8cc9-38de96c8f347\") " pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:28.572269 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.572214    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:28.608453 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.608419    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:28.701744 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.701717    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:28.702876 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:28.702851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb462075b_8a03_419f_8cc9_38de96c8f347.slice/crio-9311663a16474c664aee88d580d2b382d6f214dea70e1e4feb39fba9ee18e8f5 WatchSource:0}: Error finding container 9311663a16474c664aee88d580d2b382d6f214dea70e1e4feb39fba9ee18e8f5: Status 404 returned error can't find the container with id 9311663a16474c664aee88d580d2b382d6f214dea70e1e4feb39fba9ee18e8f5
Jul 28 20:31:28.812849 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.812814    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hdshq"]
Jul 28 20:31:28.817133 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.817117    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:28.824278 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.824114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hdshq"]
Jul 28 20:31:28.886823 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.886790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjrmt\" (UniqueName: \"kubernetes.io/projected/3ea85666-dc43-4f69-aef0-46777c0dd0bf-kube-api-access-fjrmt\") pod \"kuadrant-operator-catalog-hdshq\" (UID: \"3ea85666-dc43-4f69-aef0-46777c0dd0bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:28.987720 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.987624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjrmt\" (UniqueName: \"kubernetes.io/projected/3ea85666-dc43-4f69-aef0-46777c0dd0bf-kube-api-access-fjrmt\") pod \"kuadrant-operator-catalog-hdshq\" (UID: \"3ea85666-dc43-4f69-aef0-46777c0dd0bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:28.996214 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:28.996190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjrmt\" (UniqueName: \"kubernetes.io/projected/3ea85666-dc43-4f69-aef0-46777c0dd0bf-kube-api-access-fjrmt\") pod \"kuadrant-operator-catalog-hdshq\" (UID: \"3ea85666-dc43-4f69-aef0-46777c0dd0bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:29.102023 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:29.101989    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" event={"ID":"b462075b-8a03-419f-8cc9-38de96c8f347","Type":"ContainerStarted","Data":"9311663a16474c664aee88d580d2b382d6f214dea70e1e4feb39fba9ee18e8f5"}
Jul 28 20:31:29.127459 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:29.127428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:29.265387 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:29.265332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hdshq"]
Jul 28 20:31:29.271950 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:29.271920    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea85666_dc43_4f69_aef0_46777c0dd0bf.slice/crio-3b55a3013e3f2fa11480458fa37c8b8123146b54aba64df234dd3c65c187e348 WatchSource:0}: Error finding container 3b55a3013e3f2fa11480458fa37c8b8123146b54aba64df234dd3c65c187e348: Status 404 returned error can't find the container with id 3b55a3013e3f2fa11480458fa37c8b8123146b54aba64df234dd3c65c187e348
Jul 28 20:31:30.110208 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:30.110166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hdshq" event={"ID":"3ea85666-dc43-4f69-aef0-46777c0dd0bf","Type":"ContainerStarted","Data":"3b55a3013e3f2fa11480458fa37c8b8123146b54aba64df234dd3c65c187e348"}
Jul 28 20:31:31.116473 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.116431    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" event={"ID":"b462075b-8a03-419f-8cc9-38de96c8f347","Type":"ContainerStarted","Data":"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"}
Jul 28 20:31:31.116932 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.116477    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" podUID="b462075b-8a03-419f-8cc9-38de96c8f347" containerName="registry-server" containerID="cri-o://52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83" gracePeriod=2
Jul 28 20:31:31.117940 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.117916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hdshq" event={"ID":"3ea85666-dc43-4f69-aef0-46777c0dd0bf","Type":"ContainerStarted","Data":"66145dade83490456db9b70ce177fadbe4bbca7de10dfc4968dc67b9a5b10d19"}
Jul 28 20:31:31.144427 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.144385    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" podStartSLOduration=1.151919999 podStartE2EDuration="3.144346262s" totalImagesPullingTime="1.992426263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:31.142833163 +0000 UTC m=+386.327937869" watchObservedRunningTime="2026-07-28 20:31:31.144346262 +0000 UTC m=+386.329450968"
Jul 28 20:31:31.170815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.170762    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hdshq" podStartSLOduration=1.746437479 podStartE2EDuration="3.170743173s" totalImagesPullingTime="1.424305694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:31.169592513 +0000 UTC m=+386.354697216" watchObservedRunningTime="2026-07-28 20:31:31.170743173 +0000 UTC m=+386.355847933"
Jul 28 20:31:31.353382 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.353341    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:31.408230 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.408140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4dcp\" (UniqueName: \"kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp\") pod \"b462075b-8a03-419f-8cc9-38de96c8f347\" (UID: \"b462075b-8a03-419f-8cc9-38de96c8f347\") "
Jul 28 20:31:31.410315 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.410282    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp" (OuterVolumeSpecName: "kube-api-access-w4dcp") pod "b462075b-8a03-419f-8cc9-38de96c8f347" (UID: "b462075b-8a03-419f-8cc9-38de96c8f347"). InnerVolumeSpecName "kube-api-access-w4dcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:31.509706 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:31.509652    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4dcp\" (UniqueName: \"kubernetes.io/projected/b462075b-8a03-419f-8cc9-38de96c8f347-kube-api-access-w4dcp\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:32.122869 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.122835    2576 generic.go:356] "Generic (PLEG): container finished" podID="b462075b-8a03-419f-8cc9-38de96c8f347" containerID="52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83" exitCode=0
Jul 28 20:31:32.123375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.122876    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm"
Jul 28 20:31:32.123375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.122918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" event={"ID":"b462075b-8a03-419f-8cc9-38de96c8f347","Type":"ContainerDied","Data":"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"}
Jul 28 20:31:32.123375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.122978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5tgcm" event={"ID":"b462075b-8a03-419f-8cc9-38de96c8f347","Type":"ContainerDied","Data":"9311663a16474c664aee88d580d2b382d6f214dea70e1e4feb39fba9ee18e8f5"}
Jul 28 20:31:32.123375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.123011    2576 scope.go:117] "RemoveContainer" containerID="52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"
Jul 28 20:31:32.133053 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.133033    2576 scope.go:117] "RemoveContainer" containerID="52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"
Jul 28 20:31:32.133296 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:31:32.133281    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83\": container with ID starting with 52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83 not found: ID does not exist" containerID="52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"
Jul 28 20:31:32.133335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.133307    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83"} err="failed to get container status \"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83\": rpc error: code = NotFound desc = could not find container \"52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83\": container with ID starting with 52fc00f51fa42557b560c6b1cceac4678325ae17dab79d026ee32ba4991c2f83 not found: ID does not exist"
Jul 28 20:31:32.145583 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.145557    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:32.151910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:32.151890    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5tgcm"]
Jul 28 20:31:33.299794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:33.299759    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b462075b-8a03-419f-8cc9-38de96c8f347" path="/var/lib/kubelet/pods/b462075b-8a03-419f-8cc9-38de96c8f347/volumes"
Jul 28 20:31:39.128053 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:39.128010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:39.128506 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:39.128081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:39.149822 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:39.149791    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:39.174496 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:39.174470    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hdshq"
Jul 28 20:31:43.437763 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.437729    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"]
Jul 28 20:31:43.438144 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.438032    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b462075b-8a03-419f-8cc9-38de96c8f347" containerName="registry-server"
Jul 28 20:31:43.438144 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.438043    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b462075b-8a03-419f-8cc9-38de96c8f347" containerName="registry-server"
Jul 28 20:31:43.438144 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.438102    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b462075b-8a03-419f-8cc9-38de96c8f347" containerName="registry-server"
Jul 28 20:31:43.442640 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.442620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.445476 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.445447    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qvrsz\""
Jul 28 20:31:43.453247 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.453226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"]
Jul 28 20:31:43.508774 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.508746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.508951 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.508800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.508951 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.508819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k7jr\" (UniqueName: \"kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.610183 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.610145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.610334 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.610211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.610334 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.610239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k7jr\" (UniqueName: \"kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.610601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.610585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.610663 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.610640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.619690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.619669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k7jr\" (UniqueName: \"kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.752239 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.752198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:43.856071 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.856041    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"]
Jul 28 20:31:43.860844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.860820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:43.879018 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.878973    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"]
Jul 28 20:31:43.898658 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.898633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"]
Jul 28 20:31:43.900529 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:43.900504    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c79d7f_8295_45f3_a0cb_cb562cff5e93.slice/crio-880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438 WatchSource:0}: Error finding container 880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438: Status 404 returned error can't find the container with id 880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438
Jul 28 20:31:43.913222 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.913199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:43.913313 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.913253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkk2q\" (UniqueName: \"kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:43.913416 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:43.913331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.014559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.014490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkk2q\" (UniqueName: \"kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.014559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.014538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.014764 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.014602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.014979 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.014959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.015055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.014979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.024753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.024728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkk2q\" (UniqueName: \"kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.171394 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.171331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:44.177042 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.177015    2576 generic.go:356] "Generic (PLEG): container finished" podID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerID="8ddfcbea423c26eb63202549c62a07c6f64a290d524e53da2d85cdfff2a291b2" exitCode=0
Jul 28 20:31:44.177145 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.177082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9" event={"ID":"87c79d7f-8295-45f3-a0cb-cb562cff5e93","Type":"ContainerDied","Data":"8ddfcbea423c26eb63202549c62a07c6f64a290d524e53da2d85cdfff2a291b2"}
Jul 28 20:31:44.177145 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.177126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9" event={"ID":"87c79d7f-8295-45f3-a0cb-cb562cff5e93","Type":"ContainerStarted","Data":"880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438"}
Jul 28 20:31:44.238048 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.238018    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"]
Jul 28 20:31:44.242753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.242731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.250873 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.250459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"]
Jul 28 20:31:44.315297 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.315273    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"]
Jul 28 20:31:44.316632 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:44.316605    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67119a4f_33d7_427b_8fe6_19bf9af34a2a.slice/crio-e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f WatchSource:0}: Error finding container e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f: Status 404 returned error can't find the container with id e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f
Jul 28 20:31:44.317646 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.317625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.317757 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.317735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jllmq\" (UniqueName: \"kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.317815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.317784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.418796 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.418756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.418945 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.418812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jllmq\" (UniqueName: \"kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.418945 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.418839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.419189 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.419166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.419189 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.419182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.428685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.428659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jllmq\" (UniqueName: \"kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.559760 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.559725    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:44.710809 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:44.710779    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd11f4b_5436_40a3_9205_7dded322e886.slice/crio-7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f WatchSource:0}: Error finding container 7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f: Status 404 returned error can't find the container with id 7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f
Jul 28 20:31:44.710945 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.710862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"]
Jul 28 20:31:44.714310 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.714288    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"]
Jul 28 20:31:44.719677 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.719657    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.734913 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.734886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"]
Jul 28 20:31:44.822944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.822873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.822944 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.822934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rbc\" (UniqueName: \"kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.823141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.822976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.923323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.923298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.923449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.923343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rbc\" (UniqueName: \"kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.923449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.923380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.923686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.923671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.923754 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.923736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:44.937618 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:44.937591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rbc\" (UniqueName: \"kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:45.036155 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.036128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:45.185009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.184905    2576 generic.go:356] "Generic (PLEG): container finished" podID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerID="eb94bc79c4b07a0a0f805e53dcde4f3ec6388ada7eef99e6475045ac9999359b" exitCode=0
Jul 28 20:31:45.185193 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.185063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9" event={"ID":"87c79d7f-8295-45f3-a0cb-cb562cff5e93","Type":"ContainerDied","Data":"eb94bc79c4b07a0a0f805e53dcde4f3ec6388ada7eef99e6475045ac9999359b"}
Jul 28 20:31:45.186858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.186817    2576 generic.go:356] "Generic (PLEG): container finished" podID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerID="35fb36cdca449c4fe8f1c256f04ddd489b224cea51670e56548c1aa8bdc5605d" exitCode=0
Jul 28 20:31:45.186938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.186854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerDied","Data":"35fb36cdca449c4fe8f1c256f04ddd489b224cea51670e56548c1aa8bdc5605d"}
Jul 28 20:31:45.186938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.186902    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerStarted","Data":"e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f"}
Jul 28 20:31:45.187302 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.187276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"]
Jul 28 20:31:45.188615 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.188596    2576 generic.go:356] "Generic (PLEG): container finished" podID="bfd11f4b-5436-40a3-9205-7dded322e886" containerID="facc85d3e249b66d34c3d678abfd480d73ff55ab7105dbd698cb848a33cbf913" exitCode=0
Jul 28 20:31:45.188698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.188637    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerDied","Data":"facc85d3e249b66d34c3d678abfd480d73ff55ab7105dbd698cb848a33cbf913"}
Jul 28 20:31:45.188698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:45.188670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerStarted","Data":"7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f"}
Jul 28 20:31:45.190131 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:31:45.190112    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e331bf8_bff4_46e0_b25d_7a0a730ccdc3.slice/crio-3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59 WatchSource:0}: Error finding container 3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59: Status 404 returned error can't find the container with id 3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59
Jul 28 20:31:46.196082 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.196037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerStarted","Data":"3ee73dae988487b07f924371ad2dd5c00032a9b9aec84093425f261cbc163edb"}
Jul 28 20:31:46.198481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.198420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerStarted","Data":"8dc866567801fc74f9fa7f9d46cbd921fd3753644b7047ad00f3ca8bad6098bc"}
Jul 28 20:31:46.200138 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.200107    2576 generic.go:356] "Generic (PLEG): container finished" podID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerID="765d7f51baceb5559641ea23e0675eedcea368b50c45e3d96cf04c607df15f64" exitCode=0
Jul 28 20:31:46.200263 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.200246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc" event={"ID":"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3","Type":"ContainerDied","Data":"765d7f51baceb5559641ea23e0675eedcea368b50c45e3d96cf04c607df15f64"}
Jul 28 20:31:46.200326 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.200287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc" event={"ID":"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3","Type":"ContainerStarted","Data":"3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59"}
Jul 28 20:31:46.203320 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.203291    2576 generic.go:356] "Generic (PLEG): container finished" podID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerID="bff3ca72d1ab18f8b2f0e482191185779c7072fb62c9029476e0a80e4aef6aa5" exitCode=0
Jul 28 20:31:46.203441 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:46.203348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9" event={"ID":"87c79d7f-8295-45f3-a0cb-cb562cff5e93","Type":"ContainerDied","Data":"bff3ca72d1ab18f8b2f0e482191185779c7072fb62c9029476e0a80e4aef6aa5"}
Jul 28 20:31:47.210569 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.210533    2576 generic.go:356] "Generic (PLEG): container finished" podID="bfd11f4b-5436-40a3-9205-7dded322e886" containerID="8dc866567801fc74f9fa7f9d46cbd921fd3753644b7047ad00f3ca8bad6098bc" exitCode=0
Jul 28 20:31:47.211009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.210622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerDied","Data":"8dc866567801fc74f9fa7f9d46cbd921fd3753644b7047ad00f3ca8bad6098bc"}
Jul 28 20:31:47.212554 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.212529    2576 generic.go:356] "Generic (PLEG): container finished" podID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerID="c41bcb17f8244741dd67dd1c66d0f11c3b432f9fb35e8f5bc28f63791e68faf5" exitCode=0
Jul 28 20:31:47.212661 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.212555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc" event={"ID":"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3","Type":"ContainerDied","Data":"c41bcb17f8244741dd67dd1c66d0f11c3b432f9fb35e8f5bc28f63791e68faf5"}
Jul 28 20:31:47.214458 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.214439    2576 generic.go:356] "Generic (PLEG): container finished" podID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerID="3ee73dae988487b07f924371ad2dd5c00032a9b9aec84093425f261cbc163edb" exitCode=0
Jul 28 20:31:47.214563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.214482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerDied","Data":"3ee73dae988487b07f924371ad2dd5c00032a9b9aec84093425f261cbc163edb"}
Jul 28 20:31:47.365058 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.365032    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:47.442664 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.442630    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle\") pod \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") "
Jul 28 20:31:47.442831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.442682    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util\") pod \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") "
Jul 28 20:31:47.442831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.442780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k7jr\" (UniqueName: \"kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr\") pod \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\" (UID: \"87c79d7f-8295-45f3-a0cb-cb562cff5e93\") "
Jul 28 20:31:47.443495 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.443462    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle" (OuterVolumeSpecName: "bundle") pod "87c79d7f-8295-45f3-a0cb-cb562cff5e93" (UID: "87c79d7f-8295-45f3-a0cb-cb562cff5e93"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:47.444936 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.444907    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr" (OuterVolumeSpecName: "kube-api-access-6k7jr") pod "87c79d7f-8295-45f3-a0cb-cb562cff5e93" (UID: "87c79d7f-8295-45f3-a0cb-cb562cff5e93"). InnerVolumeSpecName "kube-api-access-6k7jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:47.449758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.449731    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util" (OuterVolumeSpecName: "util") pod "87c79d7f-8295-45f3-a0cb-cb562cff5e93" (UID: "87c79d7f-8295-45f3-a0cb-cb562cff5e93"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:47.543309 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.543271    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k7jr\" (UniqueName: \"kubernetes.io/projected/87c79d7f-8295-45f3-a0cb-cb562cff5e93-kube-api-access-6k7jr\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:47.543309 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.543300    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:47.543309 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:47.543309    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87c79d7f-8295-45f3-a0cb-cb562cff5e93-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:48.221871 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.221839    2576 generic.go:356] "Generic (PLEG): container finished" podID="bfd11f4b-5436-40a3-9205-7dded322e886" containerID="df8469a7181da73ef515aef561b6853126c53989f45ccedb131df82755d14c4a" exitCode=0
Jul 28 20:31:48.222335 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.221925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerDied","Data":"df8469a7181da73ef515aef561b6853126c53989f45ccedb131df82755d14c4a"}
Jul 28 20:31:48.223821 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.223798    2576 generic.go:356] "Generic (PLEG): container finished" podID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerID="b6ee9b6fb243d2f18f02940f9bc8aa6c7d978644bf5cd075a66245304e59eefa" exitCode=0
Jul 28 20:31:48.223924 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.223877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc" event={"ID":"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3","Type":"ContainerDied","Data":"b6ee9b6fb243d2f18f02940f9bc8aa6c7d978644bf5cd075a66245304e59eefa"}
Jul 28 20:31:48.225621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.225598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9" event={"ID":"87c79d7f-8295-45f3-a0cb-cb562cff5e93","Type":"ContainerDied","Data":"880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438"}
Jul 28 20:31:48.225715 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.225639    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="880ec6d2a2f6c0b7ee508d5dc0fed1dfb5b104ee47ae8da3416977796d6c2438"
Jul 28 20:31:48.225715 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.225605    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9"
Jul 28 20:31:48.227707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.227684    2576 generic.go:356] "Generic (PLEG): container finished" podID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerID="8a805e28d6f35eb42be7d6db57927dc7c0f08d01c235a48164b8cf667b0c790b" exitCode=0
Jul 28 20:31:48.227783 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:48.227714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerDied","Data":"8a805e28d6f35eb42be7d6db57927dc7c0f08d01c235a48164b8cf667b0c790b"}
Jul 28 20:31:49.392405 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.392381    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:49.429706 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.429678    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:49.431700 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.431683    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:49.460560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.460538    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util\") pod \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") "
Jul 28 20:31:49.460688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.460567    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8rbc\" (UniqueName: \"kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc\") pod \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") "
Jul 28 20:31:49.460688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.460645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle\") pod \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\" (UID: \"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3\") "
Jul 28 20:31:49.461206 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.461167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle" (OuterVolumeSpecName: "bundle") pod "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" (UID: "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.462687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.462664    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc" (OuterVolumeSpecName: "kube-api-access-s8rbc") pod "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" (UID: "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3"). InnerVolumeSpecName "kube-api-access-s8rbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:49.466798 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.466769    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util" (OuterVolumeSpecName: "util") pod "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" (UID: "0e331bf8-bff4-46e0-b25d-7a0a730ccdc3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.561734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jllmq\" (UniqueName: \"kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq\") pod \"bfd11f4b-5436-40a3-9205-7dded322e886\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") "
Jul 28 20:31:49.561914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561762    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle\") pod \"bfd11f4b-5436-40a3-9205-7dded322e886\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") "
Jul 28 20:31:49.561914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561786    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util\") pod \"bfd11f4b-5436-40a3-9205-7dded322e886\" (UID: \"bfd11f4b-5436-40a3-9205-7dded322e886\") "
Jul 28 20:31:49.561914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561814    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkk2q\" (UniqueName: \"kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q\") pod \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") "
Jul 28 20:31:49.561914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle\") pod \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") "
Jul 28 20:31:49.561914 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.561858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util\") pod \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\" (UID: \"67119a4f-33d7-427b-8fe6-19bf9af34a2a\") "
Jul 28 20:31:49.562151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.562110    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.562151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.562129    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.562151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.562141    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8rbc\" (UniqueName: \"kubernetes.io/projected/0e331bf8-bff4-46e0-b25d-7a0a730ccdc3-kube-api-access-s8rbc\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.562328 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.562301    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle" (OuterVolumeSpecName: "bundle") pod "bfd11f4b-5436-40a3-9205-7dded322e886" (UID: "bfd11f4b-5436-40a3-9205-7dded322e886"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.562429 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.562404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle" (OuterVolumeSpecName: "bundle") pod "67119a4f-33d7-427b-8fe6-19bf9af34a2a" (UID: "67119a4f-33d7-427b-8fe6-19bf9af34a2a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.563942 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.563921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq" (OuterVolumeSpecName: "kube-api-access-jllmq") pod "bfd11f4b-5436-40a3-9205-7dded322e886" (UID: "bfd11f4b-5436-40a3-9205-7dded322e886"). InnerVolumeSpecName "kube-api-access-jllmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:49.564232 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.564213    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q" (OuterVolumeSpecName: "kube-api-access-vkk2q") pod "67119a4f-33d7-427b-8fe6-19bf9af34a2a" (UID: "67119a4f-33d7-427b-8fe6-19bf9af34a2a"). InnerVolumeSpecName "kube-api-access-vkk2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:49.567709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.567681    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util" (OuterVolumeSpecName: "util") pod "67119a4f-33d7-427b-8fe6-19bf9af34a2a" (UID: "67119a4f-33d7-427b-8fe6-19bf9af34a2a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.568547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.568529    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util" (OuterVolumeSpecName: "util") pod "bfd11f4b-5436-40a3-9205-7dded322e886" (UID: "bfd11f4b-5436-40a3-9205-7dded322e886"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:31:49.663497 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663465    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jllmq\" (UniqueName: \"kubernetes.io/projected/bfd11f4b-5436-40a3-9205-7dded322e886-kube-api-access-jllmq\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.663497 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663493    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.663497 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663503    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd11f4b-5436-40a3-9205-7dded322e886-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.663685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663512    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkk2q\" (UniqueName: \"kubernetes.io/projected/67119a4f-33d7-427b-8fe6-19bf9af34a2a-kube-api-access-vkk2q\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.663685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663521    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-bundle\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:49.663685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:49.663529    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67119a4f-33d7-427b-8fe6-19bf9af34a2a-util\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:31:50.238792 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.238755    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p"
Jul 28 20:31:50.238792 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.238765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p" event={"ID":"67119a4f-33d7-427b-8fe6-19bf9af34a2a","Type":"ContainerDied","Data":"e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f"}
Jul 28 20:31:50.239017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.238809    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e898a01bf02ee16593f0798de067b907b02c0e642429d8cd3ac413e1ca874b6f"
Jul 28 20:31:50.240576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.240552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs" event={"ID":"bfd11f4b-5436-40a3-9205-7dded322e886","Type":"ContainerDied","Data":"7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f"}
Jul 28 20:31:50.240576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.240565    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs"
Jul 28 20:31:50.240716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.240589    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b9fd4ab29cb9542cfd8b2014825b569e91aaf61776080c4d86b5baccf70036f"
Jul 28 20:31:50.242380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.242350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc" event={"ID":"0e331bf8-bff4-46e0-b25d-7a0a730ccdc3","Type":"ContainerDied","Data":"3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59"}
Jul 28 20:31:50.242380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.242359    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc"
Jul 28 20:31:50.242502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:50.242394    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aff8176feb1a1912e6050cddecc626e8926ea4b3e8a0d18dc36c419e2078d59"
Jul 28 20:31:56.689832 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:56.689795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:57.241327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:31:57.241299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:00.422114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422081    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"]
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422406    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422417    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422424    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="extract"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422429    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="extract"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422439    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422446    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422453    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422459    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422465    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="extract"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422470    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="extract"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422480    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422485    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422492    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422498    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422503    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422508    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="pull"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422521    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="util"
Jul 28 20:32:00.422516 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422526    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="util"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422533    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422539    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422544    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422549    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422554    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="pull"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422559    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="pull"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422608    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="67119a4f-33d7-427b-8fe6-19bf9af34a2a" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422616    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd11f4b-5436-40a3-9205-7dded322e886" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422625    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c79d7f-8295-45f3-a0cb-cb562cff5e93" containerName="extract"
Jul 28 20:32:00.423051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.422631    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e331bf8-bff4-46e0-b25d-7a0a730ccdc3" containerName="extract"
Jul 28 20:32:00.425773 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.425757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:00.428124 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.428097    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bj7l9\""
Jul 28 20:32:00.438287 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.438262    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"]
Jul 28 20:32:00.550759 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.550722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w544\" (UniqueName: \"kubernetes.io/projected/c3b12d86-225c-4e16-a9a2-c21431769eed-kube-api-access-6w544\") pod \"authorino-operator-75cc8d5cb7-5v9gj\" (UID: \"c3b12d86-225c-4e16-a9a2-c21431769eed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:00.651314 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.651272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w544\" (UniqueName: \"kubernetes.io/projected/c3b12d86-225c-4e16-a9a2-c21431769eed-kube-api-access-6w544\") pod \"authorino-operator-75cc8d5cb7-5v9gj\" (UID: \"c3b12d86-225c-4e16-a9a2-c21431769eed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:00.667175 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.667150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w544\" (UniqueName: \"kubernetes.io/projected/c3b12d86-225c-4e16-a9a2-c21431769eed-kube-api-access-6w544\") pod \"authorino-operator-75cc8d5cb7-5v9gj\" (UID: \"c3b12d86-225c-4e16-a9a2-c21431769eed\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:00.736139 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.736063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:00.885925 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:00.885901    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"]
Jul 28 20:32:00.888540 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:00.888513    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b12d86_225c_4e16_a9a2_c21431769eed.slice/crio-6b76588a4a62c435281283784b65dc6c1aab27406614d4a207052fe4df6b4528 WatchSource:0}: Error finding container 6b76588a4a62c435281283784b65dc6c1aab27406614d4a207052fe4df6b4528: Status 404 returned error can't find the container with id 6b76588a4a62c435281283784b65dc6c1aab27406614d4a207052fe4df6b4528
Jul 28 20:32:01.299711 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:01.299678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj" event={"ID":"c3b12d86-225c-4e16-a9a2-c21431769eed","Type":"ContainerStarted","Data":"6b76588a4a62c435281283784b65dc6c1aab27406614d4a207052fe4df6b4528"}
Jul 28 20:32:02.321829 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:02.321791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:04.209501 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.209466    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"]
Jul 28 20:32:04.212871 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.212853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:04.216544 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.216525    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vfnrg\""
Jul 28 20:32:04.216879 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.216867    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 20:32:04.234700 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.234667    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"]
Jul 28 20:32:04.316184 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.316142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj" event={"ID":"c3b12d86-225c-4e16-a9a2-c21431769eed","Type":"ContainerStarted","Data":"a3f6d34bdebad704207fe70625b538375d45eeb8193f7199bae2834bc8711753"}
Jul 28 20:32:04.316339 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.316289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:04.355262 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.355206    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj" podStartSLOduration=1.768263948 podStartE2EDuration="4.355187012s" totalImagesPullingTime="2.586923064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:04.353265887 +0000 UTC m=+419.538370593" watchObservedRunningTime="2026-07-28 20:32:04.355187012 +0000 UTC m=+419.540291718"
Jul 28 20:32:04.387470 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.387429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgzsm\" (UniqueName: \"kubernetes.io/projected/eb62b356-3ee3-429b-90b3-b7e7edefc339-kube-api-access-qgzsm\") pod \"dns-operator-controller-manager-864dd78c4c-979gz\" (UID: \"eb62b356-3ee3-429b-90b3-b7e7edefc339\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:04.488673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.488576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgzsm\" (UniqueName: \"kubernetes.io/projected/eb62b356-3ee3-429b-90b3-b7e7edefc339-kube-api-access-qgzsm\") pod \"dns-operator-controller-manager-864dd78c4c-979gz\" (UID: \"eb62b356-3ee3-429b-90b3-b7e7edefc339\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:04.547178 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.547140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgzsm\" (UniqueName: \"kubernetes.io/projected/eb62b356-3ee3-429b-90b3-b7e7edefc339-kube-api-access-qgzsm\") pod \"dns-operator-controller-manager-864dd78c4c-979gz\" (UID: \"eb62b356-3ee3-429b-90b3-b7e7edefc339\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:04.748724 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.748684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:04.823151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:04.823118    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:05.063731 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:05.063644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"]
Jul 28 20:32:05.066862 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:05.066824    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb62b356_3ee3_429b_90b3_b7e7edefc339.slice/crio-a7fd884727f1baea509e24162bcdee3993f22dac7d74b4d7f8a95f83da365ee6 WatchSource:0}: Error finding container a7fd884727f1baea509e24162bcdee3993f22dac7d74b4d7f8a95f83da365ee6: Status 404 returned error can't find the container with id a7fd884727f1baea509e24162bcdee3993f22dac7d74b4d7f8a95f83da365ee6
Jul 28 20:32:05.321682 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:05.321591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz" event={"ID":"eb62b356-3ee3-429b-90b3-b7e7edefc339","Type":"ContainerStarted","Data":"a7fd884727f1baea509e24162bcdee3993f22dac7d74b4d7f8a95f83da365ee6"}
Jul 28 20:32:05.540298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:05.540260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:06.010090 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:06.009999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:07.160586 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:07.160518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:07.382263 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:07.382243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 20:32:08.338601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:08.338559    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz" event={"ID":"eb62b356-3ee3-429b-90b3-b7e7edefc339","Type":"ContainerStarted","Data":"6566fe4750856e934ca88bb31bba0a6dcb710adf3aa5a05d77d06a9df2b77663"}
Jul 28 20:32:08.339054 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:08.338657    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:08.378503 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:08.378430    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz" podStartSLOduration=2.071936481 podStartE2EDuration="4.378414933s" totalImagesPullingTime="2.306478452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:08.377070586 +0000 UTC m=+423.562175292" watchObservedRunningTime="2026-07-28 20:32:08.378414933 +0000 UTC m=+423.563519638"
Jul 28 20:32:09.599741 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.599701    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"]
Jul 28 20:32:09.602949 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.602934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.605565 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.605545    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9kqgb\""
Jul 28 20:32:09.630876 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.630846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"]
Jul 28 20:32:09.736234 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.736199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhkqz\" (UniqueName: \"kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.736438 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.736286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.837045 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.837008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhkqz\" (UniqueName: \"kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.837200 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.837073    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.837436 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.837420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.847570 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.847540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhkqz\" (UniqueName: \"kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:09.912639 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:09.912564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:10.093902 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:10.093813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:10.094311 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:10.094257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c04f2ef_4f3a_4e47_b9b3_067c67a6604d.slice/crio-4432fbc76964592c0713cf0fde8d77897573e368feb39f0f8fb16735dc329951 WatchSource:0}: Error finding container 4432fbc76964592c0713cf0fde8d77897573e368feb39f0f8fb16735dc329951: Status 404 returned error can't find the container with id 4432fbc76964592c0713cf0fde8d77897573e368feb39f0f8fb16735dc329951
Jul 28 20:32:10.096388 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:10.096342    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"]
Jul 28 20:32:10.351225 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:10.351184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7" event={"ID":"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d","Type":"ContainerStarted","Data":"4432fbc76964592c0713cf0fde8d77897573e368feb39f0f8fb16735dc329951"}
Jul 28 20:32:11.132525 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.132488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:11.795775 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.795734    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"]
Jul 28 20:32:11.799217 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.799187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:11.809948 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.809920    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-d9pn8\""
Jul 28 20:32:11.817688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.817664    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"]
Jul 28 20:32:11.854331 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.854298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg67j\" (UniqueName: \"kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j\") pod \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" (UID: \"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:11.954938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.954895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg67j\" (UniqueName: \"kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j\") pod \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" (UID: \"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:11.977211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:11.977178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg67j\" (UniqueName: \"kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j\") pod \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" (UID: \"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:12.113099 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:12.113010    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:12.268190 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:12.268166    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"]
Jul 28 20:32:12.270126 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:12.270099    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fdf908_43e8_4fbd_b6ff_9bc9e00fafda.slice/crio-7788b84d76091d2c40b709c797d77a49fdbbf1597f6ae86b653e074601f63919 WatchSource:0}: Error finding container 7788b84d76091d2c40b709c797d77a49fdbbf1597f6ae86b653e074601f63919: Status 404 returned error can't find the container with id 7788b84d76091d2c40b709c797d77a49fdbbf1597f6ae86b653e074601f63919
Jul 28 20:32:12.361946 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:12.361916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb" event={"ID":"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda","Type":"ContainerStarted","Data":"7788b84d76091d2c40b709c797d77a49fdbbf1597f6ae86b653e074601f63919"}
Jul 28 20:32:13.328323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:13.328269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:13.453492 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:13.453456    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:13.548601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:13.548567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:14.376062 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:14.376023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb" event={"ID":"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda","Type":"ContainerStarted","Data":"bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b"}
Jul 28 20:32:14.376509 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:14.376149    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:14.402156 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:14.402097    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb" podStartSLOduration=1.70282662 podStartE2EDuration="3.402079718s" totalImagesPullingTime="1.699253098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:14.399840588 +0000 UTC m=+429.584945294" watchObservedRunningTime="2026-07-28 20:32:14.402079718 +0000 UTC m=+429.587184523"
Jul 28 20:32:15.323881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:15.323849    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5v9gj"
Jul 28 20:32:16.999057 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:16.999007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:17.103401 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:17.103345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:17.394324 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:17.394232    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7" event={"ID":"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d","Type":"ContainerStarted","Data":"eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da"}
Jul 28 20:32:17.394498 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:17.394397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:19.278753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.278700    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7" podStartSLOduration=3.702253818 podStartE2EDuration="10.278682154s" totalImagesPullingTime="6.576428336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:17.432893743 +0000 UTC m=+432.617998451" watchObservedRunningTime="2026-07-28 20:32:19.278682154 +0000 UTC m=+434.463786863"
Jul 28 20:32:19.280226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.280207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"]
Jul 28 20:32:19.283862 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.283836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:19.284712 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.284697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.287121 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.287099    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qvrsz\""
Jul 28 20:32:19.287121 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.287116    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 20:32:19.287271 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.287099    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 20:32:19.294033 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.294012    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"]
Jul 28 20:32:19.313537 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.313508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47j5v\" (UniqueName: \"kubernetes.io/projected/3e757894-4a7a-4833-a51e-91cd5a99b98d-kube-api-access-47j5v\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.313684 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.313593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e757894-4a7a-4833-a51e-91cd5a99b98d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.313684 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.313647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.346302 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.346272    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-979gz"
Jul 28 20:32:19.414385 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.414336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.414574 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:19.414500    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 20:32:19.414574 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.414538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47j5v\" (UniqueName: \"kubernetes.io/projected/3e757894-4a7a-4833-a51e-91cd5a99b98d-kube-api-access-47j5v\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.414574 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:19.414569    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert podName:3e757894-4a7a-4833-a51e-91cd5a99b98d nodeName:}" failed. No retries permitted until 2026-07-28 20:32:19.914549544 +0000 UTC m=+435.099654233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-rgws2" (UID: "3e757894-4a7a-4833-a51e-91cd5a99b98d") : secret "plugin-serving-cert" not found
Jul 28 20:32:19.414759 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.414734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e757894-4a7a-4833-a51e-91cd5a99b98d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.415579 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.415555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e757894-4a7a-4833-a51e-91cd5a99b98d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.424216 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.424186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47j5v\" (UniqueName: \"kubernetes.io/projected/3e757894-4a7a-4833-a51e-91cd5a99b98d-kube-api-access-47j5v\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.918823 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:19.918788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:19.919013 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:19.918938    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 20:32:19.919013 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:19.919003    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert podName:3e757894-4a7a-4833-a51e-91cd5a99b98d nodeName:}" failed. No retries permitted until 2026-07-28 20:32:20.918986917 +0000 UTC m=+436.104091601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-rgws2" (UID: "3e757894-4a7a-4833-a51e-91cd5a99b98d") : secret "plugin-serving-cert" not found
Jul 28 20:32:20.110184 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:20.110153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:20.926171 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:20.926136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:20.928517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:20.928495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e757894-4a7a-4833-a51e-91cd5a99b98d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rgws2\" (UID: \"3e757894-4a7a-4833-a51e-91cd5a99b98d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:21.094491 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:21.094454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"
Jul 28 20:32:21.236408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:21.236320    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2"]
Jul 28 20:32:21.247396 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:21.247355    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e757894_4a7a_4833_a51e_91cd5a99b98d.slice/crio-09edd2c8dc8b29d9618cd0618bd5b11350a5f9a166dfc832fd3aa2b39e006cce WatchSource:0}: Error finding container 09edd2c8dc8b29d9618cd0618bd5b11350a5f9a166dfc832fd3aa2b39e006cce: Status 404 returned error can't find the container with id 09edd2c8dc8b29d9618cd0618bd5b11350a5f9a166dfc832fd3aa2b39e006cce
Jul 28 20:32:21.415407 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:21.415349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2" event={"ID":"3e757894-4a7a-4833-a51e-91cd5a99b98d","Type":"ContainerStarted","Data":"09edd2c8dc8b29d9618cd0618bd5b11350a5f9a166dfc832fd3aa2b39e006cce"}
Jul 28 20:32:21.640781 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:21.640751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:21.762063 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:21.762025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:25.385766 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:25.385736    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:28.401224 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:28.401137    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:28.674574 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:28.674485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:28.803432 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:28.803396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:30.934890 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.934852    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"]
Jul 28 20:32:30.935479 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.935133    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" containerName="manager" containerID="cri-o://eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da" gracePeriod=2
Jul 28 20:32:30.938145 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:30.938109    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:30.939636 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.939575    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"]
Jul 28 20:32:30.960070 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.959933    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"]
Jul 28 20:32:30.960667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.960184    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" containerName="manager" containerID="cri-o://bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b" gracePeriod=2
Jul 28 20:32:30.962637 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:30.962602    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:30.977547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.977472    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"]
Jul 28 20:32:30.980340 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980318    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:30.980834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980814    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" containerName="manager"
Jul 28 20:32:30.980834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980835    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" containerName="manager"
Jul 28 20:32:30.980975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" containerName="manager"
Jul 28 20:32:30.980975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980869    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" containerName="manager"
Jul 28 20:32:30.981075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.980985    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" containerName="manager"
Jul 28 20:32:30.981075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.981000    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" containerName="manager"
Jul 28 20:32:30.984197 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.984180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:30.987644 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:30.987609    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:30.995630 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:30.995576    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:30.998754 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:30.997993    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"]
Jul 28 20:32:31.001952 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.001934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:31.004758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.004732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:31.017034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.017005    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"]
Jul 28 20:32:31.021895 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:31.021856    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:31.023668 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:31.023639    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:31.121462 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.121423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs6g2\" (UniqueName: \"kubernetes.io/projected/7155e9c6-3e38-4a52-bf08-61a29e326819-kube-api-access-cs6g2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx84\" (UID: \"7155e9c6-3e38-4a52-bf08-61a29e326819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:31.121601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.121468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.121601 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.121561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff2qk\" (UniqueName: \"kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.223113 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.223027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs6g2\" (UniqueName: \"kubernetes.io/projected/7155e9c6-3e38-4a52-bf08-61a29e326819-kube-api-access-cs6g2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx84\" (UID: \"7155e9c6-3e38-4a52-bf08-61a29e326819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:31.223113 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.223068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.223323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.223144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff2qk\" (UniqueName: \"kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.223568 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.223535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.239015 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.238984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff2qk\" (UniqueName: \"kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-h87kg\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.239136 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.239040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs6g2\" (UniqueName: \"kubernetes.io/projected/7155e9c6-3e38-4a52-bf08-61a29e326819-kube-api-access-cs6g2\") pod \"limitador-operator-controller-manager-859d7b78bd-lfx84\" (UID: \"7155e9c6-3e38-4a52-bf08-61a29e326819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:31.385254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.385219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:31.393655 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:31.393631    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:32.530273 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:32.530228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:33.616408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:33.616349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:33.816412 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:33.816358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:35.531486 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:35.531442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:35.643737 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:35.643668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:40.670517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:40.670424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:40.761970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:40.761924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:42.266794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:42.266755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:43.487148 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.487118    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.489722 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.489686    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.494392 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.494335    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.498227 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.498087    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.500253 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.500220    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.500573 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.500555    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:43.508499 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:43.508476    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da54eba_841f_47e9_bb9d_92b243727b6a.slice/crio-7e6268322fa77b34c9cb01b773f293fa70baa8d783c18dee1c709cf58b8dc6e6 WatchSource:0}: Error finding container 7e6268322fa77b34c9cb01b773f293fa70baa8d783c18dee1c709cf58b8dc6e6: Status 404 returned error can't find the container with id 7e6268322fa77b34c9cb01b773f293fa70baa8d783c18dee1c709cf58b8dc6e6
Jul 28 20:32:43.520936 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.520912    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"]
Jul 28 20:32:43.523774 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:43.523745    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7155e9c6_3e38_4a52_bf08_61a29e326819.slice/crio-65eb6b5a0d9643ba4d92082fe27784f16fec17729846f9abc98abf71a0a4f818 WatchSource:0}: Error finding container 65eb6b5a0d9643ba4d92082fe27784f16fec17729846f9abc98abf71a0a4f818: Status 404 returned error can't find the container with id 65eb6b5a0d9643ba4d92082fe27784f16fec17729846f9abc98abf71a0a4f818
Jul 28 20:32:43.558087 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.558060    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84" event={"ID":"7155e9c6-3e38-4a52-bf08-61a29e326819","Type":"ContainerStarted","Data":"65eb6b5a0d9643ba4d92082fe27784f16fec17729846f9abc98abf71a0a4f818"}
Jul 28 20:32:43.559481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.559456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" event={"ID":"8da54eba-841f-47e9-bb9d-92b243727b6a","Type":"ContainerStarted","Data":"7e6268322fa77b34c9cb01b773f293fa70baa8d783c18dee1c709cf58b8dc6e6"}
Jul 28 20:32:43.560685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.560662    2576 generic.go:356] "Generic (PLEG): container finished" podID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" containerID="eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da" exitCode=0
Jul 28 20:32:43.560783 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.560708    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.560783 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.560762    2576 scope.go:117] "RemoveContainer" containerID="eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da"
Jul 28 20:32:43.562127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.562106    2576 generic.go:356] "Generic (PLEG): container finished" podID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" containerID="bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b" exitCode=0
Jul 28 20:32:43.562218 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.562148    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.562815 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.562789    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.563833 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.563808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2" event={"ID":"3e757894-4a7a-4833-a51e-91cd5a99b98d","Type":"ContainerStarted","Data":"d04766fa9d23b621f6d1e6c0b3209003e749cc96321d724ea4942aaa3d25f2fc"}
Jul 28 20:32:43.564643 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.564616    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.570721 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.570696    2576 scope.go:117] "RemoveContainer" containerID="eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da"
Jul 28 20:32:43.571009 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.570984    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da\": container with ID starting with eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da not found: ID does not exist" containerID="eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da"
Jul 28 20:32:43.571093 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.571030    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da"} err="failed to get container status \"eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da\": rpc error: code = NotFound desc = could not find container \"eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da\": container with ID starting with eef42aea3c1d43fbc1531c570389b29b097c193ec56f9d185a659f1b05bd13da not found: ID does not exist"
Jul 28 20:32:43.571093 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.571055    2576 scope.go:117] "RemoveContainer" containerID="bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b"
Jul 28 20:32:43.578586 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.578571    2576 scope.go:117] "RemoveContainer" containerID="bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b"
Jul 28 20:32:43.578867 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.578850    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b\": container with ID starting with bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b not found: ID does not exist" containerID="bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b"
Jul 28 20:32:43.578913 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.578877    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b"} err="failed to get container status \"bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b\": rpc error: code = NotFound desc = could not find container \"bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b\": container with ID starting with bf0345f5453095c09b38950ca759180794b2d0546b1c682b44ec91c3fe50040b not found: ID does not exist"
Jul 28 20:32:43.583262 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.583221    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rgws2" podStartSLOduration=2.393214433 podStartE2EDuration="24.583209109s" totalImagesPullingTime="22.189994676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:43.581407398 +0000 UTC m=+458.766512100" watchObservedRunningTime="2026-07-28 20:32:43.583209109 +0000 UTC m=+458.768313814"
Jul 28 20:32:43.583442 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.583418    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.586291 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.586269    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.648235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.648198    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume\") pod \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") "
Jul 28 20:32:43.648402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.648291    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg67j\" (UniqueName: \"kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j\") pod \"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda\" (UID: \"d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda\") "
Jul 28 20:32:43.648402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.648337    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhkqz\" (UniqueName: \"kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz\") pod \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\" (UID: \"6c04f2ef-4f3a-4e47-b9b3-067c67a6604d\") "
Jul 28 20:32:43.648843 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.648810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" (UID: "6c04f2ef-4f3a-4e47-b9b3-067c67a6604d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:43.650693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.650659    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz" (OuterVolumeSpecName: "kube-api-access-dhkqz") pod "6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" (UID: "6c04f2ef-4f3a-4e47-b9b3-067c67a6604d"). InnerVolumeSpecName "kube-api-access-dhkqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:43.650776 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.650697    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j" (OuterVolumeSpecName: "kube-api-access-wg67j") pod "d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" (UID: "d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda"). InnerVolumeSpecName "kube-api-access-wg67j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:43.749124 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.749086    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-extensions-socket-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:32:43.749124 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.749122    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg67j\" (UniqueName: \"kubernetes.io/projected/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda-kube-api-access-wg67j\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:32:43.749319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:43.749137    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhkqz\" (UniqueName: \"kubernetes.io/projected/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d-kube-api-access-dhkqz\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:32:43.875615 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.875584    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:43.877616 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.877590    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.879428 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.879407    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:43.881179 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:43.881159    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:44.576733 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.576692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84" event={"ID":"7155e9c6-3e38-4a52-bf08-61a29e326819","Type":"ContainerStarted","Data":"d5db3df4443053e98f27d51511851d29600c0a94a77e69ee304312b316bf6232"}
Jul 28 20:32:44.577226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.576791    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:44.578393 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.578346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" event={"ID":"8da54eba-841f-47e9-bb9d-92b243727b6a","Type":"ContainerStarted","Data":"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"}
Jul 28 20:32:44.578526 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.578417    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:44.578932 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:44.578908    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:44.580793 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:44.580769    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:44.598127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.598078    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84" podStartSLOduration=14.598063749 podStartE2EDuration="14.598063749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:44.596769879 +0000 UTC m=+459.781874588" watchObservedRunningTime="2026-07-28 20:32:44.598063749 +0000 UTC m=+459.783168454"
Jul 28 20:32:44.598726 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:44.598696    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:44.600412 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:44.600386    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:44.622784 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:44.622728    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" podStartSLOduration=14.622711414 podStartE2EDuration="14.622711414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:44.620610384 +0000 UTC m=+459.805715092" watchObservedRunningTime="2026-07-28 20:32:44.622711414 +0000 UTC m=+459.807816121"
Jul 28 20:32:45.300732 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:45.300689    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2mvhb\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2mvhb"
Jul 28 20:32:45.301481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:45.301460    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" path="/var/lib/kubelet/pods/6c04f2ef-4f3a-4e47-b9b3-067c67a6604d/volumes"
Jul 28 20:32:45.301803 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:45.301790    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda" path="/var/lib/kubelet/pods/d3fdf908-43e8-4fbd-b6ff-9bc9e00fafda/volumes"
Jul 28 20:32:45.302676 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:45.302656    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-dk6s7\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="6c04f2ef-4f3a-4e47-b9b3-067c67a6604d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dk6s7"
Jul 28 20:32:46.568791 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.568755    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:46.589067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.589020    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" podUID="8da54eba-841f-47e9-bb9d-92b243727b6a" containerName="manager" containerID="cri-o://2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f" gracePeriod=10
Jul 28 20:32:46.793817 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.793788    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:32:46.796410 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.796375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:46.817730 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.817706    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:32:46.845994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.845971    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:46.978494 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.978459    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume\") pod \"8da54eba-841f-47e9-bb9d-92b243727b6a\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") "
Jul 28 20:32:46.978665 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.978530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff2qk\" (UniqueName: \"kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk\") pod \"8da54eba-841f-47e9-bb9d-92b243727b6a\" (UID: \"8da54eba-841f-47e9-bb9d-92b243727b6a\") "
Jul 28 20:32:46.978665 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.978649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:46.978738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.978694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncds\" (UniqueName: \"kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:46.978847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.978825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8da54eba-841f-47e9-bb9d-92b243727b6a" (UID: "8da54eba-841f-47e9-bb9d-92b243727b6a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:32:46.980633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:46.980616    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk" (OuterVolumeSpecName: "kube-api-access-ff2qk") pod "8da54eba-841f-47e9-bb9d-92b243727b6a" (UID: "8da54eba-841f-47e9-bb9d-92b243727b6a"). InnerVolumeSpecName "kube-api-access-ff2qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:32:47.079781 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.079683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.079781 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.079742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncds\" (UniqueName: \"kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.079781 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.079774    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff2qk\" (UniqueName: \"kubernetes.io/projected/8da54eba-841f-47e9-bb9d-92b243727b6a-kube-api-access-ff2qk\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:32:47.079781 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.079785    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8da54eba-841f-47e9-bb9d-92b243727b6a-extensions-socket-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:32:47.080089 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.080068    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.101950 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.101924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncds\" (UniqueName: \"kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds\") pod \"kuadrant-operator-controller-manager-86878c769f-bxc5k\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.106797 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.106762    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.246281 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.246250    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:32:47.249025 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:32:47.248997    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9622d85_600f_458b_b854_5ac305096e17.slice/crio-8653ca4af89c6150dd763a2b41f66f74d4b5237982ed7a870cabd9a96f982bfc WatchSource:0}: Error finding container 8653ca4af89c6150dd763a2b41f66f74d4b5237982ed7a870cabd9a96f982bfc: Status 404 returned error can't find the container with id 8653ca4af89c6150dd763a2b41f66f74d4b5237982ed7a870cabd9a96f982bfc
Jul 28 20:32:47.494901 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.494861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:47.595376 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.595322    2576 generic.go:356] "Generic (PLEG): container finished" podID="8da54eba-841f-47e9-bb9d-92b243727b6a" containerID="2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f" exitCode=0
Jul 28 20:32:47.595834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.595403    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"
Jul 28 20:32:47.595834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.595400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" event={"ID":"8da54eba-841f-47e9-bb9d-92b243727b6a","Type":"ContainerDied","Data":"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"}
Jul 28 20:32:47.595834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.595481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg" event={"ID":"8da54eba-841f-47e9-bb9d-92b243727b6a","Type":"ContainerDied","Data":"7e6268322fa77b34c9cb01b773f293fa70baa8d783c18dee1c709cf58b8dc6e6"}
Jul 28 20:32:47.595834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.595496    2576 scope.go:117] "RemoveContainer" containerID="2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"
Jul 28 20:32:47.597345 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.597319    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" event={"ID":"b9622d85-600f-458b-b854-5ac305096e17","Type":"ContainerStarted","Data":"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"}
Jul 28 20:32:47.597463 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.597426    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:32:47.597547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.597463    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" event={"ID":"b9622d85-600f-458b-b854-5ac305096e17","Type":"ContainerStarted","Data":"8653ca4af89c6150dd763a2b41f66f74d4b5237982ed7a870cabd9a96f982bfc"}
Jul 28 20:32:47.607165 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.607128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:47.613142 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.613119    2576 scope.go:117] "RemoveContainer" containerID="2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"
Jul 28 20:32:47.613408 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:32:47.613389    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f\": container with ID starting with 2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f not found: ID does not exist" containerID="2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"
Jul 28 20:32:47.613461 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.613417    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f"} err="failed to get container status \"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f\": rpc error: code = NotFound desc = could not find container \"2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f\": container with ID starting with 2d8858ae8312b53ff421b2ff690167e544d948f8fb1f24b3fde61fdbe5fc2f2f not found: ID does not exist"
Jul 28 20:32:47.640688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.640277    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" podStartSLOduration=1.64025946 podStartE2EDuration="1.64025946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:47.639243902 +0000 UTC m=+462.824348608" watchObservedRunningTime="2026-07-28 20:32:47.64025946 +0000 UTC m=+462.825364168"
Jul 28 20:32:47.672459 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.672426    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:47.679585 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:47.679549    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-h87kg"]
Jul 28 20:32:49.300743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:49.300711    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8da54eba-841f-47e9-bb9d-92b243727b6a" path="/var/lib/kubelet/pods/8da54eba-841f-47e9-bb9d-92b243727b6a/volumes"
Jul 28 20:32:55.586531 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:55.586500    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lfx84"
Jul 28 20:32:58.605409 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:32:58.605358    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:33:03.108261 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:03.108225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:03.258141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:03.258106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:15.688051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:15.688011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:15.750484 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:15.750448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:17.233754 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:17.233716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:17.298596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:17.298557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:19.202595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.202551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:19.211418 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.211391    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:19.211738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.211725    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8da54eba-841f-47e9-bb9d-92b243727b6a" containerName="manager"
Jul 28 20:33:19.211791 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.211739    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8da54eba-841f-47e9-bb9d-92b243727b6a" containerName="manager"
Jul 28 20:33:19.211829 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.211805    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8da54eba-841f-47e9-bb9d-92b243727b6a" containerName="manager"
Jul 28 20:33:19.250874 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.250842    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:19.251037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.250938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.254143 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.254114    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 20:33:19.309033 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.308999    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:19.324219 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.324184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:19.365984 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.365949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.366174 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.366065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dnhc\" (UniqueName: \"kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.436283 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.436250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:19.467410 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.467296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dnhc\" (UniqueName: \"kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.467559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.467426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.468094 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.468072    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.476544 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.476517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dnhc\" (UniqueName: \"kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc\") pod \"limitador-limitador-64c94fd64f-s5mkw\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.561632 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.561601    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:19.697637 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.697609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:19.699710 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:19.699678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0a7d43_176a_445a_9199_82a6bc46b9d5.slice/crio-e768a6816eb7f4f7ddd42b297f6b086db92369bd514d8a1f7bc906a95d841b6f WatchSource:0}: Error finding container e768a6816eb7f4f7ddd42b297f6b086db92369bd514d8a1f7bc906a95d841b6f: Status 404 returned error can't find the container with id e768a6816eb7f4f7ddd42b297f6b086db92369bd514d8a1f7bc906a95d841b6f
Jul 28 20:33:19.763702 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:19.763666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" event={"ID":"3c0a7d43-176a-445a-9199-82a6bc46b9d5","Type":"ContainerStarted","Data":"e768a6816eb7f4f7ddd42b297f6b086db92369bd514d8a1f7bc906a95d841b6f"}
Jul 28 20:33:20.239087 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.239000    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:20.243515 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.243494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:20.248284 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.248263    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9qw52\""
Jul 28 20:33:20.264064 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.264031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:20.314924 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.314890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:20.377533 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.377484    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7z9\" (UniqueName: \"kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9\") pod \"authorino-58cd955675-4vx4c\" (UID: \"33dda8e9-8feb-4a62-aee7-e637ef1ecc49\") " pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:20.434505 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.434479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:20.478673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.478639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7z9\" (UniqueName: \"kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9\") pod \"authorino-58cd955675-4vx4c\" (UID: \"33dda8e9-8feb-4a62-aee7-e637ef1ecc49\") " pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:20.489712 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.489646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7z9\" (UniqueName: \"kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9\") pod \"authorino-58cd955675-4vx4c\" (UID: \"33dda8e9-8feb-4a62-aee7-e637ef1ecc49\") " pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:20.553467 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.553432    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:20.662558 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.662525    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:20.666337 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.666314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:20.685058 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.685033    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:20.708025 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.707989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:20.715584 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:20.715556    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33dda8e9_8feb_4a62_aee7_e637ef1ecc49.slice/crio-8546b2aaf0c87cee74eece951ea7aa0b6a3dfd439218191b283125f40ac8157e WatchSource:0}: Error finding container 8546b2aaf0c87cee74eece951ea7aa0b6a3dfd439218191b283125f40ac8157e: Status 404 returned error can't find the container with id 8546b2aaf0c87cee74eece951ea7aa0b6a3dfd439218191b283125f40ac8157e
Jul 28 20:33:20.773323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.773284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4vx4c" event={"ID":"33dda8e9-8feb-4a62-aee7-e637ef1ecc49","Type":"ContainerStarted","Data":"8546b2aaf0c87cee74eece951ea7aa0b6a3dfd439218191b283125f40ac8157e"}
Jul 28 20:33:20.781958 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.781889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql7n9\" (UniqueName: \"kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9\") pod \"authorino-75d556bd77-ztzqw\" (UID: \"bf5936d5-fbcf-4dd9-9570-4357748a99b1\") " pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:20.882709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.882667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql7n9\" (UniqueName: \"kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9\") pod \"authorino-75d556bd77-ztzqw\" (UID: \"bf5936d5-fbcf-4dd9-9570-4357748a99b1\") " pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:20.902753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.902718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql7n9\" (UniqueName: \"kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9\") pod \"authorino-75d556bd77-ztzqw\" (UID: \"bf5936d5-fbcf-4dd9-9570-4357748a99b1\") " pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:20.976332 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:20.976300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:21.143417 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:21.143383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:21.144077 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:21.144047    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf5936d5_fbcf_4dd9_9570_4357748a99b1.slice/crio-317c9d046be0755f45e08639c6afe4236596ea1760163bd37833f24ab8adafd9 WatchSource:0}: Error finding container 317c9d046be0755f45e08639c6afe4236596ea1760163bd37833f24ab8adafd9: Status 404 returned error can't find the container with id 317c9d046be0755f45e08639c6afe4236596ea1760163bd37833f24ab8adafd9
Jul 28 20:33:21.783144 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:21.783080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ztzqw" event={"ID":"bf5936d5-fbcf-4dd9-9570-4357748a99b1","Type":"ContainerStarted","Data":"317c9d046be0755f45e08639c6afe4236596ea1760163bd37833f24ab8adafd9"}
Jul 28 20:33:25.810549 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.810513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" event={"ID":"3c0a7d43-176a-445a-9199-82a6bc46b9d5","Type":"ContainerStarted","Data":"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"}
Jul 28 20:33:25.811036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.810576    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:25.812080 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.812056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4vx4c" event={"ID":"33dda8e9-8feb-4a62-aee7-e637ef1ecc49","Type":"ContainerStarted","Data":"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"}
Jul 28 20:33:25.813470 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.813447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ztzqw" event={"ID":"bf5936d5-fbcf-4dd9-9570-4357748a99b1","Type":"ContainerStarted","Data":"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"}
Jul 28 20:33:25.829777 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.829730    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" podStartSLOduration=1.214497279 podStartE2EDuration="6.829717954s" totalImagesPullingTime="5.615220675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:25.827857945 +0000 UTC m=+501.012962654" watchObservedRunningTime="2026-07-28 20:33:25.829717954 +0000 UTC m=+501.014822693"
Jul 28 20:33:25.842105 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.842058    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4vx4c" podStartSLOduration=1.242781014 podStartE2EDuration="5.842043572s" totalImagesPullingTime="4.599262558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:25.840867496 +0000 UTC m=+501.025972205" watchObservedRunningTime="2026-07-28 20:33:25.842043572 +0000 UTC m=+501.027148277"
Jul 28 20:33:25.856511 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.856465    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ztzqw" podStartSLOduration=1.6807464680000002 podStartE2EDuration="5.856450749s" totalImagesPullingTime="4.175704281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:25.854471574 +0000 UTC m=+501.039576281" watchObservedRunningTime="2026-07-28 20:33:25.856450749 +0000 UTC m=+501.041555455"
Jul 28 20:33:25.884434 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:25.884396    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:27.823982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:27.823924    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4vx4c" podUID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" containerName="authorino" containerID="cri-o://06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2" gracePeriod=30
Jul 28 20:33:28.073163 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.073134    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:28.251266 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.251232    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf7z9\" (UniqueName: \"kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9\") pod \"33dda8e9-8feb-4a62-aee7-e637ef1ecc49\" (UID: \"33dda8e9-8feb-4a62-aee7-e637ef1ecc49\") "
Jul 28 20:33:28.253234 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.253206    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9" (OuterVolumeSpecName: "kube-api-access-vf7z9") pod "33dda8e9-8feb-4a62-aee7-e637ef1ecc49" (UID: "33dda8e9-8feb-4a62-aee7-e637ef1ecc49"). InnerVolumeSpecName "kube-api-access-vf7z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:28.352753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.352718    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf7z9\" (UniqueName: \"kubernetes.io/projected/33dda8e9-8feb-4a62-aee7-e637ef1ecc49-kube-api-access-vf7z9\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:33:28.829621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.829583    2576 generic.go:356] "Generic (PLEG): container finished" podID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" containerID="06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2" exitCode=0
Jul 28 20:33:28.830073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.829629    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4vx4c"
Jul 28 20:33:28.830073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.829673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4vx4c" event={"ID":"33dda8e9-8feb-4a62-aee7-e637ef1ecc49","Type":"ContainerDied","Data":"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"}
Jul 28 20:33:28.830073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.829725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4vx4c" event={"ID":"33dda8e9-8feb-4a62-aee7-e637ef1ecc49","Type":"ContainerDied","Data":"8546b2aaf0c87cee74eece951ea7aa0b6a3dfd439218191b283125f40ac8157e"}
Jul 28 20:33:28.830073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.829754    2576 scope.go:117] "RemoveContainer" containerID="06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"
Jul 28 20:33:28.839136 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.839120    2576 scope.go:117] "RemoveContainer" containerID="06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"
Jul 28 20:33:28.839393 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:28.839355    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2\": container with ID starting with 06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2 not found: ID does not exist" containerID="06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"
Jul 28 20:33:28.839447 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.839405    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2"} err="failed to get container status \"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2\": rpc error: code = NotFound desc = could not find container \"06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2\": container with ID starting with 06374a0294938ac798c151e20780634f98b5aeddfe22e0bb6919764dce0367c2 not found: ID does not exist"
Jul 28 20:33:28.851808 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.851783    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:28.855475 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:28.855456    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4vx4c"]
Jul 28 20:33:29.300460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:29.300427    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" path="/var/lib/kubelet/pods/33dda8e9-8feb-4a62-aee7-e637ef1ecc49/volumes"
Jul 28 20:33:34.771681 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:34.771643    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:34.772226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:34.771822    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" podUID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" containerName="limitador" containerID="cri-o://d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd" gracePeriod=30
Jul 28 20:33:34.774331 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:34.774307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:35.716114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.716084    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:35.717530 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.717514    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dnhc\" (UniqueName: \"kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc\") pod \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") "
Jul 28 20:33:35.717587 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.717564    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file\") pod \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\" (UID: \"3c0a7d43-176a-445a-9199-82a6bc46b9d5\") "
Jul 28 20:33:35.717891 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.717872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file" (OuterVolumeSpecName: "config-file") pod "3c0a7d43-176a-445a-9199-82a6bc46b9d5" (UID: "3c0a7d43-176a-445a-9199-82a6bc46b9d5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:33:35.719458 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.719434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc" (OuterVolumeSpecName: "kube-api-access-5dnhc") pod "3c0a7d43-176a-445a-9199-82a6bc46b9d5" (UID: "3c0a7d43-176a-445a-9199-82a6bc46b9d5"). InnerVolumeSpecName "kube-api-access-5dnhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:35.818529 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.818490    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dnhc\" (UniqueName: \"kubernetes.io/projected/3c0a7d43-176a-445a-9199-82a6bc46b9d5-kube-api-access-5dnhc\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:33:35.818529 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.818523    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3c0a7d43-176a-445a-9199-82a6bc46b9d5-config-file\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:33:35.866995 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.866909    2576 generic.go:356] "Generic (PLEG): container finished" podID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" containerID="d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd" exitCode=0
Jul 28 20:33:35.866995 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.866957    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"
Jul 28 20:33:35.867194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.866995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" event={"ID":"3c0a7d43-176a-445a-9199-82a6bc46b9d5","Type":"ContainerDied","Data":"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"}
Jul 28 20:33:35.867194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.867058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5mkw" event={"ID":"3c0a7d43-176a-445a-9199-82a6bc46b9d5","Type":"ContainerDied","Data":"e768a6816eb7f4f7ddd42b297f6b086db92369bd514d8a1f7bc906a95d841b6f"}
Jul 28 20:33:35.867194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.867089    2576 scope.go:117] "RemoveContainer" containerID="d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"
Jul 28 20:33:35.876691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.876673    2576 scope.go:117] "RemoveContainer" containerID="d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"
Jul 28 20:33:35.876961 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:35.876942    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd\": container with ID starting with d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd not found: ID does not exist" containerID="d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"
Jul 28 20:33:35.877014 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.876971    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd"} err="failed to get container status \"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd\": rpc error: code = NotFound desc = could not find container \"d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd\": container with ID starting with d1ce10ba641c19476e18396c3f871ec276226909e271874d7d3f57782e8eacfd not found: ID does not exist"
Jul 28 20:33:35.890819 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.890794    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:35.895456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:35.895433    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5mkw"]
Jul 28 20:33:36.342760 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.342722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:36.501633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.501592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:36.775577 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.775544    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"]
Jul 28 20:33:36.775987 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.775970    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" containerName="authorino"
Jul 28 20:33:36.776037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.775989    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" containerName="authorino"
Jul 28 20:33:36.776037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.776004    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" containerName="limitador"
Jul 28 20:33:36.776037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.776009    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" containerName="limitador"
Jul 28 20:33:36.776141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.776062    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33dda8e9-8feb-4a62-aee7-e637ef1ecc49" containerName="authorino"
Jul 28 20:33:36.776141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.776074    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" containerName="limitador"
Jul 28 20:33:36.778753 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.778731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.783513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.783488    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:33:36.783634 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.783529    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 28 20:33:36.784344 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.784319    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-w6pnm\""
Jul 28 20:33:36.784344 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.784336    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:33:36.794864 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.794841    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"]
Jul 28 20:33:36.827947 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.827912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9cf8f5a7-b377-4628-93f5-6cda90204416-data\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.828290 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.827955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rk5\" (UniqueName: \"kubernetes.io/projected/9cf8f5a7-b377-4628-93f5-6cda90204416-kube-api-access-d2rk5\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.832402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.832355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:36.929001 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.928962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rk5\" (UniqueName: \"kubernetes.io/projected/9cf8f5a7-b377-4628-93f5-6cda90204416-kube-api-access-d2rk5\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.929211 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.929082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9cf8f5a7-b377-4628-93f5-6cda90204416-data\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.929510 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.929489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9cf8f5a7-b377-4628-93f5-6cda90204416-data\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:36.937806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:36.937780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rk5\" (UniqueName: \"kubernetes.io/projected/9cf8f5a7-b377-4628-93f5-6cda90204416-kube-api-access-d2rk5\") pod \"postgres-69fb4594cf-xj7l5\" (UID: \"9cf8f5a7-b377-4628-93f5-6cda90204416\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:37.089505 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:37.089412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:37.220123 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:37.220095    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"]
Jul 28 20:33:37.221835 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:37.221808    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf8f5a7_b377_4628_93f5_6cda90204416.slice/crio-2b0165a4534f113282ac774572282a5509a680b375c4b4451c62f6cc964773a2 WatchSource:0}: Error finding container 2b0165a4534f113282ac774572282a5509a680b375c4b4451c62f6cc964773a2: Status 404 returned error can't find the container with id 2b0165a4534f113282ac774572282a5509a680b375c4b4451c62f6cc964773a2
Jul 28 20:33:37.300481 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:37.300442    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0a7d43-176a-445a-9199-82a6bc46b9d5" path="/var/lib/kubelet/pods/3c0a7d43-176a-445a-9199-82a6bc46b9d5/volumes"
Jul 28 20:33:37.879455 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:37.879420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5" event={"ID":"9cf8f5a7-b377-4628-93f5-6cda90204416","Type":"ContainerStarted","Data":"2b0165a4534f113282ac774572282a5509a680b375c4b4451c62f6cc964773a2"}
Jul 28 20:33:42.915887 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:42.915797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5" event={"ID":"9cf8f5a7-b377-4628-93f5-6cda90204416","Type":"ContainerStarted","Data":"f9af42ea51dc2b723cdcea8f1e5f4aacf74bcb56ed50947dbc9af3216b4c937c"}
Jul 28 20:33:42.916265 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:42.915916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:42.933743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:42.933689    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5" podStartSLOduration=1.634266989 podStartE2EDuration="6.933672533s" totalImagesPullingTime="5.299405544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:42.932429444 +0000 UTC m=+518.117534151" watchObservedRunningTime="2026-07-28 20:33:42.933672533 +0000 UTC m=+518.118777239"
Jul 28 20:33:48.950125 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:48.950096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xj7l5"
Jul 28 20:33:49.285929 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.285894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:49.523257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.523221    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:49.529351 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.529327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:49.534688 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.534660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:49.638402 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.638355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22ps7\" (UniqueName: \"kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7\") pod \"authorino-5d4b67c446-55zhj\" (UID: \"baec8ecb-07a2-44ea-948a-9f52cd799418\") " pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:49.739698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.739658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22ps7\" (UniqueName: \"kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7\") pod \"authorino-5d4b67c446-55zhj\" (UID: \"baec8ecb-07a2-44ea-948a-9f52cd799418\") " pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:49.749066 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.749034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22ps7\" (UniqueName: \"kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7\") pod \"authorino-5d4b67c446-55zhj\" (UID: \"baec8ecb-07a2-44ea-948a-9f52cd799418\") " pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:49.800207 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.800119    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:49.800464 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.800450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:49.830206 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.830169    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:33:49.835488 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.835465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:49.838507 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.838479    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 20:33:49.841385 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.841340    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:33:49.940773 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.940734    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:49.941019 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.940998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkctd\" (UniqueName: \"kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:49.941109 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.941041    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:49.941936 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:49.941909    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaec8ecb_07a2_44ea_948a_9f52cd799418.slice/crio-3398e77bbdf0e49aa90fa3f3cb077449216648dfb5c8980b6927f6ab94a66964 WatchSource:0}: Error finding container 3398e77bbdf0e49aa90fa3f3cb077449216648dfb5c8980b6927f6ab94a66964: Status 404 returned error can't find the container with id 3398e77bbdf0e49aa90fa3f3cb077449216648dfb5c8980b6927f6ab94a66964
Jul 28 20:33:49.953633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:49.953607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-55zhj" event={"ID":"baec8ecb-07a2-44ea-948a-9f52cd799418","Type":"ContainerStarted","Data":"3398e77bbdf0e49aa90fa3f3cb077449216648dfb5c8980b6927f6ab94a66964"}
Jul 28 20:33:50.042099 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.042064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkctd\" (UniqueName: \"kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:50.042301 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.042127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:50.044603 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.044578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:50.050106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.050086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkctd\" (UniqueName: \"kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd\") pod \"authorino-676667759c-whcfq\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") " pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:50.147758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.147674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:33:50.287015 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.286982    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:33:50.288371 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:50.288334    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89221aea_5b74_4c99_919c_9eed0b7f1140.slice/crio-5e62b1ff5051cb0f3a3d2883cf84ecdcad2b0b745e2e32c7dca2f6164a121a6f WatchSource:0}: Error finding container 5e62b1ff5051cb0f3a3d2883cf84ecdcad2b0b745e2e32c7dca2f6164a121a6f: Status 404 returned error can't find the container with id 5e62b1ff5051cb0f3a3d2883cf84ecdcad2b0b745e2e32c7dca2f6164a121a6f
Jul 28 20:33:50.960428 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.960393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-55zhj" event={"ID":"baec8ecb-07a2-44ea-948a-9f52cd799418","Type":"ContainerStarted","Data":"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"}
Jul 28 20:33:50.960897 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.960499    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-55zhj" podUID="baec8ecb-07a2-44ea-948a-9f52cd799418" containerName="authorino" containerID="cri-o://10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee" gracePeriod=30
Jul 28 20:33:50.961650 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.961628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676667759c-whcfq" event={"ID":"89221aea-5b74-4c99-919c-9eed0b7f1140","Type":"ContainerStarted","Data":"5e62b1ff5051cb0f3a3d2883cf84ecdcad2b0b745e2e32c7dca2f6164a121a6f"}
Jul 28 20:33:50.976693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:50.976636    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-55zhj" podStartSLOduration=1.563601238 podStartE2EDuration="1.976619634s" totalImagesPullingTime="413.018396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:50.975196862 +0000 UTC m=+526.160301569" watchObservedRunningTime="2026-07-28 20:33:50.976619634 +0000 UTC m=+526.161724363"
Jul 28 20:33:51.108559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.108519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:51.214092 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.214033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:51.254724 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.254690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22ps7\" (UniqueName: \"kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7\") pod \"baec8ecb-07a2-44ea-948a-9f52cd799418\" (UID: \"baec8ecb-07a2-44ea-948a-9f52cd799418\") "
Jul 28 20:33:51.256706 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.256676    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7" (OuterVolumeSpecName: "kube-api-access-22ps7") pod "baec8ecb-07a2-44ea-948a-9f52cd799418" (UID: "baec8ecb-07a2-44ea-948a-9f52cd799418"). InnerVolumeSpecName "kube-api-access-22ps7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:51.355851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.355818    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22ps7\" (UniqueName: \"kubernetes.io/projected/baec8ecb-07a2-44ea-948a-9f52cd799418-kube-api-access-22ps7\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:33:51.970633 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.970597    2576 generic.go:356] "Generic (PLEG): container finished" podID="baec8ecb-07a2-44ea-948a-9f52cd799418" containerID="10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee" exitCode=0
Jul 28 20:33:51.971119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.970651    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-55zhj"
Jul 28 20:33:51.971119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.970687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-55zhj" event={"ID":"baec8ecb-07a2-44ea-948a-9f52cd799418","Type":"ContainerDied","Data":"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"}
Jul 28 20:33:51.971119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.970739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-55zhj" event={"ID":"baec8ecb-07a2-44ea-948a-9f52cd799418","Type":"ContainerDied","Data":"3398e77bbdf0e49aa90fa3f3cb077449216648dfb5c8980b6927f6ab94a66964"}
Jul 28 20:33:51.971119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.970763    2576 scope.go:117] "RemoveContainer" containerID="10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"
Jul 28 20:33:51.979311 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.979293    2576 scope.go:117] "RemoveContainer" containerID="10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"
Jul 28 20:33:51.979611 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:51.979592    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee\": container with ID starting with 10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee not found: ID does not exist" containerID="10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"
Jul 28 20:33:51.979670 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.979620    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee"} err="failed to get container status \"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee\": rpc error: code = NotFound desc = could not find container \"10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee\": container with ID starting with 10d538483669e8fac5905d7d79800ab5f266af40fc037019014aac9b8349f5ee not found: ID does not exist"
Jul 28 20:33:51.987310 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.987284    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:51.991371 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:51.991337    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-55zhj"]
Jul 28 20:33:52.977457 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:52.977419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676667759c-whcfq" event={"ID":"89221aea-5b74-4c99-919c-9eed0b7f1140","Type":"ContainerStarted","Data":"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"}
Jul 28 20:33:52.993674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:52.993623    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-676667759c-whcfq" podStartSLOduration=2.075527439 podStartE2EDuration="3.993606987s" totalImagesPullingTime="1.918079548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:33:52.991232855 +0000 UTC m=+528.176337563" watchObservedRunningTime="2026-07-28 20:33:52.993606987 +0000 UTC m=+528.178711692"
Jul 28 20:33:53.031483 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.031142    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:53.031483 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.031386    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ztzqw" podUID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" containerName="authorino" containerID="cri-o://57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f" gracePeriod=30
Jul 28 20:33:53.286036 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.286010    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:53.301008 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.300976    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baec8ecb-07a2-44ea-948a-9f52cd799418" path="/var/lib/kubelet/pods/baec8ecb-07a2-44ea-948a-9f52cd799418/volumes"
Jul 28 20:33:53.371249 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.371216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql7n9\" (UniqueName: \"kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9\") pod \"bf5936d5-fbcf-4dd9-9570-4357748a99b1\" (UID: \"bf5936d5-fbcf-4dd9-9570-4357748a99b1\") "
Jul 28 20:33:53.373241 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.373220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9" (OuterVolumeSpecName: "kube-api-access-ql7n9") pod "bf5936d5-fbcf-4dd9-9570-4357748a99b1" (UID: "bf5936d5-fbcf-4dd9-9570-4357748a99b1"). InnerVolumeSpecName "kube-api-access-ql7n9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:33:53.471945 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.471897    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql7n9\" (UniqueName: \"kubernetes.io/projected/bf5936d5-fbcf-4dd9-9570-4357748a99b1-kube-api-access-ql7n9\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:33:53.984610 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.984572    2576 generic.go:356] "Generic (PLEG): container finished" podID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" containerID="57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f" exitCode=0
Jul 28 20:33:53.985037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.984616    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ztzqw"
Jul 28 20:33:53.985037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.984658    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ztzqw" event={"ID":"bf5936d5-fbcf-4dd9-9570-4357748a99b1","Type":"ContainerDied","Data":"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"}
Jul 28 20:33:53.985037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.984703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ztzqw" event={"ID":"bf5936d5-fbcf-4dd9-9570-4357748a99b1","Type":"ContainerDied","Data":"317c9d046be0755f45e08639c6afe4236596ea1760163bd37833f24ab8adafd9"}
Jul 28 20:33:53.985037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.984726    2576 scope.go:117] "RemoveContainer" containerID="57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"
Jul 28 20:33:53.993474 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.993350    2576 scope.go:117] "RemoveContainer" containerID="57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"
Jul 28 20:33:53.993657 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:53.993639    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f\": container with ID starting with 57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f not found: ID does not exist" containerID="57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"
Jul 28 20:33:53.993698 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:53.993666    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f"} err="failed to get container status \"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f\": rpc error: code = NotFound desc = could not find container \"57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f\": container with ID starting with 57f69affc0b01acc636c05a98238db26c2571d0fa2ecec00f27cd8d6a80b960f not found: ID does not exist"
Jul 28 20:33:54.007987 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:54.007962    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:54.012103 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:54.012085    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ztzqw"]
Jul 28 20:33:55.301325 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:55.301283    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" path="/var/lib/kubelet/pods/bf5936d5-fbcf-4dd9-9570-4357748a99b1/volumes"
Jul 28 20:33:58.000534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000453    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:33:58.000916 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000840    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" containerName="authorino"
Jul 28 20:33:58.000916 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000855    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" containerName="authorino"
Jul 28 20:33:58.000916 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000870    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baec8ecb-07a2-44ea-948a-9f52cd799418" containerName="authorino"
Jul 28 20:33:58.000916 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000876    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="baec8ecb-07a2-44ea-948a-9f52cd799418" containerName="authorino"
Jul 28 20:33:58.001056 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000956    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5936d5-fbcf-4dd9-9570-4357748a99b1" containerName="authorino"
Jul 28 20:33:58.001056 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.000965    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="baec8ecb-07a2-44ea-948a-9f52cd799418" containerName="authorino"
Jul 28 20:33:58.014801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.014771    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.014971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.014842    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:33:58.017328 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.017302    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mlgl6\""
Jul 28 20:33:58.017502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.017327    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 20:33:58.017502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.017303    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 20:33:58.023188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.023106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:58.088016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.087977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:58.113225 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.113188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.113390 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.113232    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdcvv\" (UniqueName: \"kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.192582 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.192547    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:33:58.210806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.210774    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:33:58.210976 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.210880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.213675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.213649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.213790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.213682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcvv\" (UniqueName: \"kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.213843 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:58.213800    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 28 20:33:58.213881 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:33:58.213856    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert podName:30d2a042-87f8-4eb7-b595-cae956bd3fbf nodeName:}" failed. No retries permitted until 2026-07-28 20:33:58.713839932 +0000 UTC m=+533.898944621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert") pod "maas-controller-67fb85b555-vvp9t" (UID: "30d2a042-87f8-4eb7-b595-cae956bd3fbf") : secret "maas-controller-webhook-cert" not found
Jul 28 20:33:58.222406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.222355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdcvv\" (UniqueName: \"kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.314543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.314447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nffhn\" (UniqueName: \"kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.314711 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.314574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.411008 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.410968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:33:58.415986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.415965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.416081 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.416037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nffhn\" (UniqueName: \"kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.418582 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.418557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.425672 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.425648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nffhn\" (UniqueName: \"kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn\") pod \"maas-controller-57b5564d8b-5pftp\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") " pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.522735 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.522697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:33:58.683174 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.683138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:33:58.686407 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:58.686382    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6617e02_8c8c_49d7_afdf_7737e20493c6.slice/crio-09d95b7f3ff85b5806c18cc4723a6c79ad84fb1721737938c1006de86cfcf7f7 WatchSource:0}: Error finding container 09d95b7f3ff85b5806c18cc4723a6c79ad84fb1721737938c1006de86cfcf7f7: Status 404 returned error can't find the container with id 09d95b7f3ff85b5806c18cc4723a6c79ad84fb1721737938c1006de86cfcf7f7
Jul 28 20:33:58.718244 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.718212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.720642 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.720613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") pod \"maas-controller-67fb85b555-vvp9t\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") " pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:58.926895 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:58.926801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:33:59.013180 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:59.013150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-5pftp" event={"ID":"b6617e02-8c8c-49d7-afdf-7737e20493c6","Type":"ContainerStarted","Data":"09d95b7f3ff85b5806c18cc4723a6c79ad84fb1721737938c1006de86cfcf7f7"}
Jul 28 20:33:59.082436 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:33:59.082412    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:33:59.083988 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:33:59.083965    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d2a042_87f8_4eb7_b595_cae956bd3fbf.slice/crio-e9c3bbeb03212267219c4e92b80afdff9ca8656e75d6302c12bb5c830d6cb447 WatchSource:0}: Error finding container e9c3bbeb03212267219c4e92b80afdff9ca8656e75d6302c12bb5c830d6cb447: Status 404 returned error can't find the container with id e9c3bbeb03212267219c4e92b80afdff9ca8656e75d6302c12bb5c830d6cb447
Jul 28 20:34:00.021665 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:00.021617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fb85b555-vvp9t" event={"ID":"30d2a042-87f8-4eb7-b595-cae956bd3fbf","Type":"ContainerStarted","Data":"e9c3bbeb03212267219c4e92b80afdff9ca8656e75d6302c12bb5c830d6cb447"}
Jul 28 20:34:01.090975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:01.090906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:01.191892 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:01.191860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:01.297215 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:01.297116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:02.035547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.035508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-5pftp" event={"ID":"b6617e02-8c8c-49d7-afdf-7737e20493c6","Type":"ContainerStarted","Data":"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"}
Jul 28 20:34:02.035757 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.035741    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:34:02.037096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.037068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fb85b555-vvp9t" event={"ID":"30d2a042-87f8-4eb7-b595-cae956bd3fbf","Type":"ContainerStarted","Data":"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"}
Jul 28 20:34:02.037229 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.037184    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:34:02.057305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.057246    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b5564d8b-5pftp" podStartSLOduration=1.725216751 podStartE2EDuration="4.057226641s" totalImagesPullingTime="2.33200989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:02.053905912 +0000 UTC m=+537.239010619" watchObservedRunningTime="2026-07-28 20:34:02.057226641 +0000 UTC m=+537.242331411"
Jul 28 20:34:02.075921 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:02.075859    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67fb85b555-vvp9t" podStartSLOduration=3.013695019 podStartE2EDuration="5.075843051s" totalImagesPullingTime="2.062148032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:02.073162308 +0000 UTC m=+537.258267015" watchObservedRunningTime="2026-07-28 20:34:02.075843051 +0000 UTC m=+537.260947759"
Jul 28 20:34:12.719425 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:12.719383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:12.823020 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:12.822983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:13.046098 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.046071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:34:13.046266 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.046198    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:34:13.127685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.127645    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:34:13.127915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.127886    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67fb85b555-vvp9t" podUID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" containerName="manager" containerID="cri-o://ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7" gracePeriod=30
Jul 28 20:34:13.401620 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.401595    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:34:13.557228 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.557135    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdcvv\" (UniqueName: \"kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv\") pod \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") "
Jul 28 20:34:13.557426 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.557254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") pod \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\" (UID: \"30d2a042-87f8-4eb7-b595-cae956bd3fbf\") "
Jul 28 20:34:13.559419 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.559391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv" (OuterVolumeSpecName: "kube-api-access-tdcvv") pod "30d2a042-87f8-4eb7-b595-cae956bd3fbf" (UID: "30d2a042-87f8-4eb7-b595-cae956bd3fbf"). InnerVolumeSpecName "kube-api-access-tdcvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:34:13.559529 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.559434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "30d2a042-87f8-4eb7-b595-cae956bd3fbf" (UID: "30d2a042-87f8-4eb7-b595-cae956bd3fbf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:34:13.658202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.658161    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdcvv\" (UniqueName: \"kubernetes.io/projected/30d2a042-87f8-4eb7-b595-cae956bd3fbf-kube-api-access-tdcvv\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:34:13.658202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:13.658194    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30d2a042-87f8-4eb7-b595-cae956bd3fbf-webhook-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:34:14.103572 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.103542    2576 generic.go:356] "Generic (PLEG): container finished" podID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" containerID="ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7" exitCode=0
Jul 28 20:34:14.103986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.103589    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fb85b555-vvp9t"
Jul 28 20:34:14.103986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.103624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fb85b555-vvp9t" event={"ID":"30d2a042-87f8-4eb7-b595-cae956bd3fbf","Type":"ContainerDied","Data":"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"}
Jul 28 20:34:14.103986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.103681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fb85b555-vvp9t" event={"ID":"30d2a042-87f8-4eb7-b595-cae956bd3fbf","Type":"ContainerDied","Data":"e9c3bbeb03212267219c4e92b80afdff9ca8656e75d6302c12bb5c830d6cb447"}
Jul 28 20:34:14.103986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.103714    2576 scope.go:117] "RemoveContainer" containerID="ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"
Jul 28 20:34:14.113517 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.113497    2576 scope.go:117] "RemoveContainer" containerID="ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"
Jul 28 20:34:14.113797 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:34:14.113780    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7\": container with ID starting with ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7 not found: ID does not exist" containerID="ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"
Jul 28 20:34:14.113840 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.113807    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7"} err="failed to get container status \"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7\": rpc error: code = NotFound desc = could not find container \"ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7\": container with ID starting with ba30256813236fb33131d1a78fa2e7c392c6d9bed89f6792618700bb42eb46f7 not found: ID does not exist"
Jul 28 20:34:14.126449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.126414    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:34:14.133305 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:14.133270    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67fb85b555-vvp9t"]
Jul 28 20:34:15.302097 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:15.302052    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" path="/var/lib/kubelet/pods/30d2a042-87f8-4eb7-b595-cae956bd3fbf/volumes"
Jul 28 20:34:32.823881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:32.823841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:32.919792 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:32.919743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:33.435882 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.435833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:33.560308 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.560270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:33.663718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.663679    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"]
Jul 28 20:34:33.664066 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.664052    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" containerName="manager"
Jul 28 20:34:33.664130 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.664069    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" containerName="manager"
Jul 28 20:34:33.664167 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.664158    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d2a042-87f8-4eb7-b595-cae956bd3fbf" containerName="manager"
Jul 28 20:34:33.670657 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.670633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.673277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.673251    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jtts8\""
Jul 28 20:34:33.674143 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.674119    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 20:34:33.675138 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.675114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"]
Jul 28 20:34:33.732235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.732149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b08c9497-a4a3-4713-a485-8678fbc1359d-maas-api-tls\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.732235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.732188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08c9497-a4a3-4713-a485-8678fbc1359d-openshift-service-ca\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.732456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.732274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpwpt\" (UniqueName: \"kubernetes.io/projected/b08c9497-a4a3-4713-a485-8678fbc1359d-kube-api-access-qpwpt\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.833802 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.833769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b08c9497-a4a3-4713-a485-8678fbc1359d-maas-api-tls\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.834189 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.833817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08c9497-a4a3-4713-a485-8678fbc1359d-openshift-service-ca\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.834189 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.833879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpwpt\" (UniqueName: \"kubernetes.io/projected/b08c9497-a4a3-4713-a485-8678fbc1359d-kube-api-access-qpwpt\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.834543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.834517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b08c9497-a4a3-4713-a485-8678fbc1359d-openshift-service-ca\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.836384 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.836346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b08c9497-a4a3-4713-a485-8678fbc1359d-maas-api-tls\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.842627 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.842590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpwpt\" (UniqueName: \"kubernetes.io/projected/b08c9497-a4a3-4713-a485-8678fbc1359d-kube-api-access-qpwpt\") pod \"maas-api-8b5db48f9-6wdg7\" (UID: \"b08c9497-a4a3-4713-a485-8678fbc1359d\") " pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:33.982862 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:33.982770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:34.002460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:34.002426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:34.134128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:34.134103    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"]
Jul 28 20:34:34.135538 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:34:34.135508    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08c9497_a4a3_4713_a485_8678fbc1359d.slice/crio-9bf4f562769c674a129b957adce9020159e9ac877bebf02c7b7e4565492911a8 WatchSource:0}: Error finding container 9bf4f562769c674a129b957adce9020159e9ac877bebf02c7b7e4565492911a8: Status 404 returned error can't find the container with id 9bf4f562769c674a129b957adce9020159e9ac877bebf02c7b7e4565492911a8
Jul 28 20:34:34.147732 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:34.147702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:34.217642 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:34.217607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7" event={"ID":"b08c9497-a4a3-4713-a485-8678fbc1359d","Type":"ContainerStarted","Data":"9bf4f562769c674a129b957adce9020159e9ac877bebf02c7b7e4565492911a8"}
Jul 28 20:34:36.230310 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:36.230277    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7" event={"ID":"b08c9497-a4a3-4713-a485-8678fbc1359d","Type":"ContainerStarted","Data":"77fbed28c95fa83d21de86d60e657a5de37dee323461a08dbe85912c36dc1b25"}
Jul 28 20:34:36.230724 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:36.230407    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:36.253115 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:36.253063    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7" podStartSLOduration=1.8415289879999999 podStartE2EDuration="3.253046856s" totalImagesPullingTime="1.411517868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:36.25139502 +0000 UTC m=+571.436499725" watchObservedRunningTime="2026-07-28 20:34:36.253046856 +0000 UTC m=+571.438151562"
Jul 28 20:34:42.241309 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:42.241279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8b5db48f9-6wdg7"
Jul 28 20:34:44.495954 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:44.495917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:44.587873 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:44.587834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:49.793215 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.793179    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"]
Jul 28 20:34:49.796896 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.796870    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.802561 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.802539    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 20:34:49.802770 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.802753    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 20:34:49.802867 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.802837    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 20:34:49.802927 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.802887    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sx8kt\""
Jul 28 20:34:49.823584 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.823557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"]
Jul 28 20:34:49.844561 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.844520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:34:49.871524 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.871686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.871686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4chf2\" (UniqueName: \"kubernetes.io/projected/4181c467-5cda-49c1-bc02-a6279a0459ae-kube-api-access-4chf2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.871686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.871834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871699    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.871834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.871717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4181c467-5cda-49c1-bc02-a6279a0459ae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972345 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4chf2\" (UniqueName: \"kubernetes.io/projected/4181c467-5cda-49c1-bc02-a6279a0459ae-kube-api-access-4chf2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4181c467-5cda-49c1-bc02-a6279a0459ae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972859 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.972895 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.972871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.974881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.974849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4181c467-5cda-49c1-bc02-a6279a0459ae-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.975090 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.975069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4181c467-5cda-49c1-bc02-a6279a0459ae-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:49.981331 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:49.981308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4chf2\" (UniqueName: \"kubernetes.io/projected/4181c467-5cda-49c1-bc02-a6279a0459ae-kube-api-access-4chf2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5\" (UID: \"4181c467-5cda-49c1-bc02-a6279a0459ae\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:50.108234 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:50.108141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:34:50.250560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:50.250530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"]
Jul 28 20:34:50.252148 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:34:50.252119    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4181c467_5cda_49c1_bc02_a6279a0459ae.slice/crio-d307ac88883970d2f6834d64542933960720132d8e539ad62f4c05910101215b WatchSource:0}: Error finding container d307ac88883970d2f6834d64542933960720132d8e539ad62f4c05910101215b: Status 404 returned error can't find the container with id d307ac88883970d2f6834d64542933960720132d8e539ad62f4c05910101215b
Jul 28 20:34:50.312033 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:50.311995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5" event={"ID":"4181c467-5cda-49c1-bc02-a6279a0459ae","Type":"ContainerStarted","Data":"d307ac88883970d2f6834d64542933960720132d8e539ad62f4c05910101215b"}
Jul 28 20:34:52.323854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:52.323817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5" event={"ID":"4181c467-5cda-49c1-bc02-a6279a0459ae","Type":"ContainerStarted","Data":"c5af35808294e0463df5927e2129c49a061a3abe839ee4f1a8a1e86d63859a54"}
Jul 28 20:34:52.346491 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:52.346439    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5" podStartSLOduration=2.276166768 podStartE2EDuration="3.346421281s" totalImagesPullingTime="1.070254513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:34:52.343505993 +0000 UTC m=+587.528610703" watchObservedRunningTime="2026-07-28 20:34:52.346421281 +0000 UTC m=+587.531525984"
Jul 28 20:34:56.646857 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:34:56.646817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:00.109100 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:00.109058    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:35:00.109595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:00.109132    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:35:00.121621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:00.121596    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:35:00.383158 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:00.383075    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5"
Jul 28 20:35:01.447253 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:01.447215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:07.943283 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:07.943241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:08.043936 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:08.043895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:11.412816 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.412780    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"]
Jul 28 20:35:11.415864 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.415846    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.418298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.418276    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 20:35:11.426127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.426105    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"]
Jul 28 20:35:11.447987 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.447956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:11.465513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465468    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.465513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81466578-4350-4bb8-afa4-fb1004e28776-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.465742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.465742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.465742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kkbk\" (UniqueName: \"kubernetes.io/projected/81466578-4350-4bb8-afa4-fb1004e28776-kube-api-access-7kkbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.465742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.465740    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566394 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kkbk\" (UniqueName: \"kubernetes.io/projected/81466578-4350-4bb8-afa4-fb1004e28776-kube-api-access-7kkbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566581 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.566755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81466578-4350-4bb8-afa4-fb1004e28776-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.567009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.567009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.566999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.567110 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.567070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.568880 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.568850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81466578-4350-4bb8-afa4-fb1004e28776-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.569021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.569007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81466578-4350-4bb8-afa4-fb1004e28776-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.574794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.574774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kkbk\" (UniqueName: \"kubernetes.io/projected/81466578-4350-4bb8-afa4-fb1004e28776-kube-api-access-7kkbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7\" (UID: \"81466578-4350-4bb8-afa4-fb1004e28776\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.727638 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.727544    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:11.870319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.870289    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"]
Jul 28 20:35:11.871557 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:35:11.871530    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81466578_4350_4bb8_afa4_fb1004e28776.slice/crio-7a2abafb750adf4477b4e89652258292eedc3a516a3aea97fc7a8c42686b15ad WatchSource:0}: Error finding container 7a2abafb750adf4477b4e89652258292eedc3a516a3aea97fc7a8c42686b15ad: Status 404 returned error can't find the container with id 7a2abafb750adf4477b4e89652258292eedc3a516a3aea97fc7a8c42686b15ad
Jul 28 20:35:11.872858 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:11.872840    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:35:12.439358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:12.439255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7" event={"ID":"81466578-4350-4bb8-afa4-fb1004e28776","Type":"ContainerStarted","Data":"5afc91791a6b30d32722b35c24d596e2f59f43af032af7b46106b170fafa3b95"}
Jul 28 20:35:12.439934 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:12.439404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7" event={"ID":"81466578-4350-4bb8-afa4-fb1004e28776","Type":"ContainerStarted","Data":"7a2abafb750adf4477b4e89652258292eedc3a516a3aea97fc7a8c42686b15ad"}
Jul 28 20:35:12.458662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:12.458606    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7" podStartSLOduration=1.181365155 podStartE2EDuration="1.458589269s" totalImagesPullingTime="277.224114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:35:12.457401347 +0000 UTC m=+607.642506044" watchObservedRunningTime="2026-07-28 20:35:12.458589269 +0000 UTC m=+607.643693975"
Jul 28 20:35:21.727842 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:21.727797    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:21.728479 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:21.727862    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:21.740653 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:21.740624    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:22.505137 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:22.505109    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7"
Jul 28 20:35:30.941859 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:30.941752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:35:50.585617 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:35:50.585580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:36:33.341650 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:36:33.341611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:36:56.175822 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:36:56.175783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:11.496905 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.496856    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9747db87b-rzcjv"]
Jul 28 20:37:11.500888 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.500864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.511576 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.511544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhlq\" (UniqueName: \"kubernetes.io/projected/5386f485-3a20-4f69-828d-73af48e84cb0-kube-api-access-bdhlq\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.511721 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.511639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5386f485-3a20-4f69-828d-73af48e84cb0-tls-cert\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.512108 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.512086    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9747db87b-rzcjv"]
Jul 28 20:37:11.612892 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.612851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhlq\" (UniqueName: \"kubernetes.io/projected/5386f485-3a20-4f69-828d-73af48e84cb0-kube-api-access-bdhlq\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.613068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.612919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5386f485-3a20-4f69-828d-73af48e84cb0-tls-cert\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.615345 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.615323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5386f485-3a20-4f69-828d-73af48e84cb0-tls-cert\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.621010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.620984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhlq\" (UniqueName: \"kubernetes.io/projected/5386f485-3a20-4f69-828d-73af48e84cb0-kube-api-access-bdhlq\") pod \"authorino-9747db87b-rzcjv\" (UID: \"5386f485-3a20-4f69-828d-73af48e84cb0\") " pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.811026 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.810934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9747db87b-rzcjv"
Jul 28 20:37:11.917278 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.917225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:11.955763 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:11.955716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9747db87b-rzcjv"]
Jul 28 20:37:11.958490 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:37:11.958466    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5386f485_3a20_4f69_828d_73af48e84cb0.slice/crio-d355de8548b9695c084e7269b2a2762b82a13b484c567864011ec5192b723e93 WatchSource:0}: Error finding container d355de8548b9695c084e7269b2a2762b82a13b484c567864011ec5192b723e93: Status 404 returned error can't find the container with id d355de8548b9695c084e7269b2a2762b82a13b484c567864011ec5192b723e93
Jul 28 20:37:12.019341 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:12.019307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:12.081179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:12.081089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9747db87b-rzcjv" event={"ID":"5386f485-3a20-4f69-828d-73af48e84cb0","Type":"ContainerStarted","Data":"d355de8548b9695c084e7269b2a2762b82a13b484c567864011ec5192b723e93"}
Jul 28 20:37:13.090673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.090634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9747db87b-rzcjv" event={"ID":"5386f485-3a20-4f69-828d-73af48e84cb0","Type":"ContainerStarted","Data":"215dcf0b618c2264f6710a15f78af6a18310ffa0c0fcbe4b70f027d14d970cce"}
Jul 28 20:37:13.113848 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.113781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9747db87b-rzcjv" podStartSLOduration=1.7791323810000002 podStartE2EDuration="2.11375697s" totalImagesPullingTime="334.624589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:37:13.110526722 +0000 UTC m=+728.295631428" watchObservedRunningTime="2026-07-28 20:37:13.11375697 +0000 UTC m=+728.298861678"
Jul 28 20:37:13.141693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.141661    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:37:13.141946 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.141879    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-676667759c-whcfq" podUID="89221aea-5b74-4c99-919c-9eed0b7f1140" containerName="authorino" containerID="cri-o://a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63" gracePeriod=30
Jul 28 20:37:13.165386 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.161490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:13.260503 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.260463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:37:13.401303 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.401278    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:37:13.429926 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.429895    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert\") pod \"89221aea-5b74-4c99-919c-9eed0b7f1140\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") "
Jul 28 20:37:13.430114 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.429968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkctd\" (UniqueName: \"kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd\") pod \"89221aea-5b74-4c99-919c-9eed0b7f1140\" (UID: \"89221aea-5b74-4c99-919c-9eed0b7f1140\") "
Jul 28 20:37:13.432021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.431994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd" (OuterVolumeSpecName: "kube-api-access-tkctd") pod "89221aea-5b74-4c99-919c-9eed0b7f1140" (UID: "89221aea-5b74-4c99-919c-9eed0b7f1140"). InnerVolumeSpecName "kube-api-access-tkctd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:37:13.443091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.443065    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "89221aea-5b74-4c99-919c-9eed0b7f1140" (UID: "89221aea-5b74-4c99-919c-9eed0b7f1140"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:37:13.531504 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.531468    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkctd\" (UniqueName: \"kubernetes.io/projected/89221aea-5b74-4c99-919c-9eed0b7f1140-kube-api-access-tkctd\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:37:13.531504 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:13.531497    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89221aea-5b74-4c99-919c-9eed0b7f1140-tls-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:37:14.099959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.099919    2576 generic.go:356] "Generic (PLEG): container finished" podID="89221aea-5b74-4c99-919c-9eed0b7f1140" containerID="a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63" exitCode=0
Jul 28 20:37:14.100640 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.100620    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676667759c-whcfq"
Jul 28 20:37:14.108510 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.108465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676667759c-whcfq" event={"ID":"89221aea-5b74-4c99-919c-9eed0b7f1140","Type":"ContainerDied","Data":"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"}
Jul 28 20:37:14.108693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.108533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676667759c-whcfq" event={"ID":"89221aea-5b74-4c99-919c-9eed0b7f1140","Type":"ContainerDied","Data":"5e62b1ff5051cb0f3a3d2883cf84ecdcad2b0b745e2e32c7dca2f6164a121a6f"}
Jul 28 20:37:14.108693 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.108556    2576 scope.go:117] "RemoveContainer" containerID="a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"
Jul 28 20:37:14.119010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.118991    2576 scope.go:117] "RemoveContainer" containerID="a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"
Jul 28 20:37:14.119354 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:37:14.119330    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63\": container with ID starting with a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63 not found: ID does not exist" containerID="a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"
Jul 28 20:37:14.119468 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.119381    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63"} err="failed to get container status \"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63\": rpc error: code = NotFound desc = could not find container \"a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63\": container with ID starting with a135951509dca04aed861424412f8628cfe36ff9d0b6412ca345f7db25d7cf63 not found: ID does not exist"
Jul 28 20:37:14.130241 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.130204    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:37:14.132596 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:14.132571    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-676667759c-whcfq"]
Jul 28 20:37:15.300815 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:15.300784    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89221aea-5b74-4c99-919c-9eed0b7f1140" path="/var/lib/kubelet/pods/89221aea-5b74-4c99-919c-9eed0b7f1140/volumes"
Jul 28 20:37:21.737441 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:37:21.737395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:38:00.985160 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:00.985119    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:38:00.985790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:00.985325    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57b5564d8b-5pftp" podUID="b6617e02-8c8c-49d7-afdf-7737e20493c6" containerName="manager" containerID="cri-o://e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3" gracePeriod=30
Jul 28 20:38:01.238723 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.238665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:38:01.260039 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.260008    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nffhn\" (UniqueName: \"kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn\") pod \"b6617e02-8c8c-49d7-afdf-7737e20493c6\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") "
Jul 28 20:38:01.260195 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.260066    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert\") pod \"b6617e02-8c8c-49d7-afdf-7737e20493c6\" (UID: \"b6617e02-8c8c-49d7-afdf-7737e20493c6\") "
Jul 28 20:38:01.262290 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.262261    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b6617e02-8c8c-49d7-afdf-7737e20493c6" (UID: "b6617e02-8c8c-49d7-afdf-7737e20493c6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:38:01.262595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.262576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn" (OuterVolumeSpecName: "kube-api-access-nffhn") pod "b6617e02-8c8c-49d7-afdf-7737e20493c6" (UID: "b6617e02-8c8c-49d7-afdf-7737e20493c6"). InnerVolumeSpecName "kube-api-access-nffhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:38:01.360803 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360772    2576 generic.go:356] "Generic (PLEG): container finished" podID="b6617e02-8c8c-49d7-afdf-7737e20493c6" containerID="e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3" exitCode=0
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360818    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-5pftp"
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-5pftp" event={"ID":"b6617e02-8c8c-49d7-afdf-7737e20493c6","Type":"ContainerDied","Data":"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"}
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360884    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nffhn\" (UniqueName: \"kubernetes.io/projected/b6617e02-8c8c-49d7-afdf-7737e20493c6-kube-api-access-nffhn\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360907    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b6617e02-8c8c-49d7-afdf-7737e20493c6-webhook-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-5pftp" event={"ID":"b6617e02-8c8c-49d7-afdf-7737e20493c6","Type":"ContainerDied","Data":"09d95b7f3ff85b5806c18cc4723a6c79ad84fb1721737938c1006de86cfcf7f7"}
Jul 28 20:38:01.360986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.360936    2576 scope.go:117] "RemoveContainer" containerID="e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"
Jul 28 20:38:01.369839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.369818    2576 scope.go:117] "RemoveContainer" containerID="e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"
Jul 28 20:38:01.370095 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:38:01.370073    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3\": container with ID starting with e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3 not found: ID does not exist" containerID="e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"
Jul 28 20:38:01.370145 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.370107    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3"} err="failed to get container status \"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3\": rpc error: code = NotFound desc = could not find container \"e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3\": container with ID starting with e6a85f2d4a5a02e8ff19ea84f6d7c7986280c69c3a12f1ff4e3ad7536f463bf3 not found: ID does not exist"
Jul 28 20:38:01.392832 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.392792    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:38:01.395272 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:01.395251    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-5pftp"]
Jul 28 20:38:03.301043 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:03.301008    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6617e02-8c8c-49d7-afdf-7737e20493c6" path="/var/lib/kubelet/pods/b6617e02-8c8c-49d7-afdf-7737e20493c6/volumes"
Jul 28 20:38:30.646739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:38:30.646604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:02.258961 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.258922    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259310    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89221aea-5b74-4c99-919c-9eed0b7f1140" containerName="authorino"
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259322    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89221aea-5b74-4c99-919c-9eed0b7f1140" containerName="authorino"
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259348    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6617e02-8c8c-49d7-afdf-7737e20493c6" containerName="manager"
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259354    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6617e02-8c8c-49d7-afdf-7737e20493c6" containerName="manager"
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259427    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6617e02-8c8c-49d7-afdf-7737e20493c6" containerName="manager"
Jul 28 20:39:02.259443 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.259434    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89221aea-5b74-4c99-919c-9eed0b7f1140" containerName="authorino"
Jul 28 20:39:02.262536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.262520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.266178 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.266152    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 20:39:02.266279 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.266178    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 20:39:02.266279 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.266243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mlgl6\""
Jul 28 20:39:02.272096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.272069    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:02.306232 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.306197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khwxd\" (UniqueName: \"kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.306440 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.306324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.407251 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.407202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.407460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.407269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khwxd\" (UniqueName: \"kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.409687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.409659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.415500 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.415469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khwxd\" (UniqueName: \"kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd\") pod \"maas-controller-57b5564d8b-qstgv\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") " pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.573477 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.573353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:02.711691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:02.711660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:02.714095 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:39:02.714068    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15205006_c090_4540_abda_9c4f8aa03d19.slice/crio-9f1b6c733b9c72485220a90e105deb569ed91ddc7e3d74ac205bb8d2abfad633 WatchSource:0}: Error finding container 9f1b6c733b9c72485220a90e105deb569ed91ddc7e3d74ac205bb8d2abfad633: Status 404 returned error can't find the container with id 9f1b6c733b9c72485220a90e105deb569ed91ddc7e3d74ac205bb8d2abfad633
Jul 28 20:39:03.698106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:03.698067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-qstgv" event={"ID":"15205006-c090-4540-abda-9c4f8aa03d19","Type":"ContainerStarted","Data":"9f1b6c733b9c72485220a90e105deb569ed91ddc7e3d74ac205bb8d2abfad633"}
Jul 28 20:39:05.710351 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:05.710311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-qstgv" event={"ID":"15205006-c090-4540-abda-9c4f8aa03d19","Type":"ContainerStarted","Data":"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"}
Jul 28 20:39:05.710735 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:05.710531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:05.729932 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:05.729881    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b5564d8b-qstgv" podStartSLOduration=1.501005148 podStartE2EDuration="3.729867058s" totalImagesPullingTime="2.22886191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:39:05.727785594 +0000 UTC m=+840.912890299" watchObservedRunningTime="2026-07-28 20:39:05.729867058 +0000 UTC m=+840.914971763"
Jul 28 20:39:16.717381 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:16.717334    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:21.293128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.293082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:21.423826 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.423787    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:21.424046 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.424002    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57b5564d8b-qstgv" podUID="15205006-c090-4540-abda-9c4f8aa03d19" containerName="manager" containerID="cri-o://ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28" gracePeriod=30
Jul 28 20:39:21.676025 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.676000    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:21.785659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.785627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khwxd\" (UniqueName: \"kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd\") pod \"15205006-c090-4540-abda-9c4f8aa03d19\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") "
Jul 28 20:39:21.785825 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.785684    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert\") pod \"15205006-c090-4540-abda-9c4f8aa03d19\" (UID: \"15205006-c090-4540-abda-9c4f8aa03d19\") "
Jul 28 20:39:21.787671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.787647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd" (OuterVolumeSpecName: "kube-api-access-khwxd") pod "15205006-c090-4540-abda-9c4f8aa03d19" (UID: "15205006-c090-4540-abda-9c4f8aa03d19"). InnerVolumeSpecName "kube-api-access-khwxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:39:21.787760 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.787745    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "15205006-c090-4540-abda-9c4f8aa03d19" (UID: "15205006-c090-4540-abda-9c4f8aa03d19"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:39:21.795870 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.795842    2576 generic.go:356] "Generic (PLEG): container finished" podID="15205006-c090-4540-abda-9c4f8aa03d19" containerID="ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28" exitCode=0
Jul 28 20:39:21.795978 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.795895    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-qstgv"
Jul 28 20:39:21.795978 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.795912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-qstgv" event={"ID":"15205006-c090-4540-abda-9c4f8aa03d19","Type":"ContainerDied","Data":"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"}
Jul 28 20:39:21.795978 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.795969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-qstgv" event={"ID":"15205006-c090-4540-abda-9c4f8aa03d19","Type":"ContainerDied","Data":"9f1b6c733b9c72485220a90e105deb569ed91ddc7e3d74ac205bb8d2abfad633"}
Jul 28 20:39:21.796141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.795992    2576 scope.go:117] "RemoveContainer" containerID="ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"
Jul 28 20:39:21.805021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.805007    2576 scope.go:117] "RemoveContainer" containerID="ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"
Jul 28 20:39:21.805273 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:39:21.805255    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28\": container with ID starting with ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28 not found: ID does not exist" containerID="ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"
Jul 28 20:39:21.805313 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.805283    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28"} err="failed to get container status \"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28\": rpc error: code = NotFound desc = could not find container \"ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28\": container with ID starting with ef2c213f9fe6b45efe0231bf828365bf0e1cbc5602ac2e3cdcc57d62ce2edd28 not found: ID does not exist"
Jul 28 20:39:21.820069 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.820046    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:21.826224 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.826199    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-qstgv"]
Jul 28 20:39:21.886659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.886584    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khwxd\" (UniqueName: \"kubernetes.io/projected/15205006-c090-4540-abda-9c4f8aa03d19-kube-api-access-khwxd\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:39:21.886659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:21.886609    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15205006-c090-4540-abda-9c4f8aa03d19-webhook-cert\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:39:22.552030 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.551994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:22.675843 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.675812    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b5564d8b-wpfd5"]
Jul 28 20:39:22.676192 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.676180    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15205006-c090-4540-abda-9c4f8aa03d19" containerName="manager"
Jul 28 20:39:22.676236 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.676194    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15205006-c090-4540-abda-9c4f8aa03d19" containerName="manager"
Jul 28 20:39:22.676276 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.676256    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15205006-c090-4540-abda-9c4f8aa03d19" containerName="manager"
Jul 28 20:39:22.680814 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.680790    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.683383 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.683343    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 20:39:22.684150 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.684131    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 20:39:22.684206 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.684165    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mlgl6\""
Jul 28 20:39:22.694051 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.694032    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-wpfd5"]
Jul 28 20:39:22.794010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.793978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbc1b530-a7c3-4913-9aab-71c044ba51f0-webhook-cert\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.794173 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.794054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd4qd\" (UniqueName: \"kubernetes.io/projected/cbc1b530-a7c3-4913-9aab-71c044ba51f0-kube-api-access-xd4qd\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.895358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.895272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbc1b530-a7c3-4913-9aab-71c044ba51f0-webhook-cert\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.895358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.895333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd4qd\" (UniqueName: \"kubernetes.io/projected/cbc1b530-a7c3-4913-9aab-71c044ba51f0-kube-api-access-xd4qd\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.897823 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.897794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbc1b530-a7c3-4913-9aab-71c044ba51f0-webhook-cert\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.903852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.903830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd4qd\" (UniqueName: \"kubernetes.io/projected/cbc1b530-a7c3-4913-9aab-71c044ba51f0-kube-api-access-xd4qd\") pod \"maas-controller-57b5564d8b-wpfd5\" (UID: \"cbc1b530-a7c3-4913-9aab-71c044ba51f0\") " pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:22.991928 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:22.991898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:23.127764 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.127728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b5564d8b-wpfd5"]
Jul 28 20:39:23.130834 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:39:23.130797    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc1b530_a7c3_4913_9aab_71c044ba51f0.slice/crio-b97c3b6a868f27f64d8ea654c7edd9b86199616e51750c59182d2195acf2d72d WatchSource:0}: Error finding container b97c3b6a868f27f64d8ea654c7edd9b86199616e51750c59182d2195acf2d72d: Status 404 returned error can't find the container with id b97c3b6a868f27f64d8ea654c7edd9b86199616e51750c59182d2195acf2d72d
Jul 28 20:39:23.300407 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.300352    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15205006-c090-4540-abda-9c4f8aa03d19" path="/var/lib/kubelet/pods/15205006-c090-4540-abda-9c4f8aa03d19/volumes"
Jul 28 20:39:23.816567 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.816531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-wpfd5" event={"ID":"cbc1b530-a7c3-4913-9aab-71c044ba51f0","Type":"ContainerStarted","Data":"c06411df04dc0a611c70e2099c1f4cad64f03cea1a20ffdf5d078b04bade7379"}
Jul 28 20:39:23.816975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.816620    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b5564d8b-wpfd5" event={"ID":"cbc1b530-a7c3-4913-9aab-71c044ba51f0","Type":"ContainerStarted","Data":"b97c3b6a868f27f64d8ea654c7edd9b86199616e51750c59182d2195acf2d72d"}
Jul 28 20:39:23.816975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.816759    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:23.842788 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:23.842731    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b5564d8b-wpfd5" podStartSLOduration=1.26074265 podStartE2EDuration="1.842705638s" totalImagesPullingTime="581.962988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:39:23.840170179 +0000 UTC m=+859.025274885" watchObservedRunningTime="2026-07-28 20:39:23.842705638 +0000 UTC m=+859.027810344"
Jul 28 20:39:34.824404 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:34.824359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b5564d8b-wpfd5"
Jul 28 20:39:57.335826 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:57.335733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:39:59.019304 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:39:59.019263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:03.989152 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:40:03.989121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:04.388563 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:40:04.388512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:04.459881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:40:04.459838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:10.730438 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:40:10.730396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:40:10.831343 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:40:10.831301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:56.287722 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:42:56.287678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:42:56.358609 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:42:56.358570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:43:28.460980 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:43:28.460943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:44:27.399990 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:27.399901    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:44:27.400514 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:27.400100    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" podUID="b9622d85-600f-458b-b854-5ac305096e17" containerName="manager" containerID="cri-o://4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985" gracePeriod=10
Jul 28 20:44:31.254738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.254712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:44:31.269136 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.269106    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume\") pod \"b9622d85-600f-458b-b854-5ac305096e17\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") "
Jul 28 20:44:31.269268 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.269176    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ncds\" (UniqueName: \"kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds\") pod \"b9622d85-600f-458b-b854-5ac305096e17\" (UID: \"b9622d85-600f-458b-b854-5ac305096e17\") "
Jul 28 20:44:31.269493 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.269467    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b9622d85-600f-458b-b854-5ac305096e17" (UID: "b9622d85-600f-458b-b854-5ac305096e17"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:44:31.271170 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.271150    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds" (OuterVolumeSpecName: "kube-api-access-6ncds") pod "b9622d85-600f-458b-b854-5ac305096e17" (UID: "b9622d85-600f-458b-b854-5ac305096e17"). InnerVolumeSpecName "kube-api-access-6ncds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:44:31.370065 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.370024    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9622d85-600f-458b-b854-5ac305096e17-extensions-socket-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:44:31.370065 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.370056    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ncds\" (UniqueName: \"kubernetes.io/projected/b9622d85-600f-458b-b854-5ac305096e17-kube-api-access-6ncds\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:44:31.490687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.490651    2576 generic.go:356] "Generic (PLEG): container finished" podID="b9622d85-600f-458b-b854-5ac305096e17" containerID="4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985" exitCode=0
Jul 28 20:44:31.490881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.490711    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"
Jul 28 20:44:31.490881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.490737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" event={"ID":"b9622d85-600f-458b-b854-5ac305096e17","Type":"ContainerDied","Data":"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"}
Jul 28 20:44:31.490881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.490830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k" event={"ID":"b9622d85-600f-458b-b854-5ac305096e17","Type":"ContainerDied","Data":"8653ca4af89c6150dd763a2b41f66f74d4b5237982ed7a870cabd9a96f982bfc"}
Jul 28 20:44:31.490881 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.490862    2576 scope.go:117] "RemoveContainer" containerID="4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"
Jul 28 20:44:31.500237 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.500216    2576 scope.go:117] "RemoveContainer" containerID="4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"
Jul 28 20:44:31.500536 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:44:31.500516    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985\": container with ID starting with 4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985 not found: ID does not exist" containerID="4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"
Jul 28 20:44:31.500620 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.500544    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985"} err="failed to get container status \"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985\": rpc error: code = NotFound desc = could not find container \"4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985\": container with ID starting with 4e1d5472de16c5394a4fe99e2ff3caab9963a5eacf513acaf2ad79630918a985 not found: ID does not exist"
Jul 28 20:44:31.517599 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.517565    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:44:31.523508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:31.523476    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bxc5k"]
Jul 28 20:44:33.300641 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:33.300607    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9622d85-600f-458b-b854-5ac305096e17" path="/var/lib/kubelet/pods/b9622d85-600f-458b-b854-5ac305096e17/volumes"
Jul 28 20:44:34.945863 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:44:34.945833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:45:00.151690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.151651    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"]
Jul 28 20:45:00.152126 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.152020    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9622d85-600f-458b-b854-5ac305096e17" containerName="manager"
Jul 28 20:45:00.152126 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.152031    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9622d85-600f-458b-b854-5ac305096e17" containerName="manager"
Jul 28 20:45:00.152126 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.152103    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9622d85-600f-458b-b854-5ac305096e17" containerName="manager"
Jul 28 20:45:00.155328 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.155312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"
Jul 28 20:45:00.167999 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.167977    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:45:00.174851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.174831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"
Jul 28 20:45:00.179429 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.179394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"]
Jul 28 20:45:00.317250 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.317221    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"]
Jul 28 20:45:00.319019 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:45:00.318992    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa26325_49e2_4a8e_9849_040b941ac1be.slice/crio-ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087 WatchSource:0}: Error finding container ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087: Status 404 returned error can't find the container with id ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087
Jul 28 20:45:00.321116 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.321093    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:45:00.649200 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:00.649156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp" event={"ID":"3fa26325-49e2-4a8e-9849-040b941ac1be","Type":"ContainerStarted","Data":"ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087"}
Jul 28 20:45:04.673067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:04.673022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp" event={"ID":"3fa26325-49e2-4a8e-9849-040b941ac1be","Type":"ContainerStarted","Data":"5fc31cf1012d1b683889292104c697f3521020b48d6f1242687d0b68d8f82fa9"}
Jul 28 20:45:04.692778 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:04.692716    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp" podStartSLOduration=0.700975406 podStartE2EDuration="4.692697801s" totalImagesPullingTime="3.991722395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:04.690465846 +0000 UTC m=+1199.875570550" watchObservedRunningTime="2026-07-28 20:45:04.692697801 +0000 UTC m=+1199.877802508"
Jul 28 20:45:05.680177 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:05.680138    2576 generic.go:356] "Generic (PLEG): container finished" podID="3fa26325-49e2-4a8e-9849-040b941ac1be" containerID="5fc31cf1012d1b683889292104c697f3521020b48d6f1242687d0b68d8f82fa9" exitCode=0
Jul 28 20:45:05.680622 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:05.680223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp" event={"ID":"3fa26325-49e2-4a8e-9849-040b941ac1be","Type":"ContainerDied","Data":"5fc31cf1012d1b683889292104c697f3521020b48d6f1242687d0b68d8f82fa9"}
Jul 28 20:45:06.822041 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:06.822007    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"
Jul 28 20:45:07.692106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:07.692020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp" event={"ID":"3fa26325-49e2-4a8e-9849-040b941ac1be","Type":"ContainerDied","Data":"ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087"}
Jul 28 20:45:07.692106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:07.692070    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca03ded4e546ff9dc2d289668c1b52fedf48951aafca17a727de41b1f7317087"
Jul 28 20:45:07.692106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:07.692037    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754525-vcmbp"
Jul 28 20:45:28.618083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.618047    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"]
Jul 28 20:45:28.620543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.618475    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa26325-49e2-4a8e-9849-040b941ac1be" containerName="cleanup"
Jul 28 20:45:28.620543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.618489    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa26325-49e2-4a8e-9849-040b941ac1be" containerName="cleanup"
Jul 28 20:45:28.620543 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.618564    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa26325-49e2-4a8e-9849-040b941ac1be" containerName="cleanup"
Jul 28 20:45:28.621451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.621436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.626377 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.626329    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9kqgb\""
Jul 28 20:45:28.637330 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.637300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"]
Jul 28 20:45:28.685972 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.685931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16fffe17-9517-4d3e-b5f1-7deda336a128-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.686154 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.686036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sg8s\" (UniqueName: \"kubernetes.io/projected/16fffe17-9517-4d3e-b5f1-7deda336a128-kube-api-access-4sg8s\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.786784 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.786741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16fffe17-9517-4d3e-b5f1-7deda336a128-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.786976 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.786851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sg8s\" (UniqueName: \"kubernetes.io/projected/16fffe17-9517-4d3e-b5f1-7deda336a128-kube-api-access-4sg8s\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.787172 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.787150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16fffe17-9517-4d3e-b5f1-7deda336a128-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.795478 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.795449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sg8s\" (UniqueName: \"kubernetes.io/projected/16fffe17-9517-4d3e-b5f1-7deda336a128-kube-api-access-4sg8s\") pod \"kuadrant-operator-controller-manager-86878c769f-7k5fb\" (UID: \"16fffe17-9517-4d3e-b5f1-7deda336a128\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:28.931575 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:28.931496    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:29.080151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:29.080120    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"]
Jul 28 20:45:29.080812 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:45:29.080784    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fffe17_9517_4d3e_b5f1_7deda336a128.slice/crio-78c4f10a42ebc847aec3e0ee57549dd318d01aeeb47c8814eebfc0f13fa12fa9 WatchSource:0}: Error finding container 78c4f10a42ebc847aec3e0ee57549dd318d01aeeb47c8814eebfc0f13fa12fa9: Status 404 returned error can't find the container with id 78c4f10a42ebc847aec3e0ee57549dd318d01aeeb47c8814eebfc0f13fa12fa9
Jul 28 20:45:29.812573 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:29.812536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb" event={"ID":"16fffe17-9517-4d3e-b5f1-7deda336a128","Type":"ContainerStarted","Data":"27549b277ec8e4bc0f87e788130279b3f2cd64c13a757bcd3ba27220c3d044d6"}
Jul 28 20:45:29.812975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:29.812593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb" event={"ID":"16fffe17-9517-4d3e-b5f1-7deda336a128","Type":"ContainerStarted","Data":"78c4f10a42ebc847aec3e0ee57549dd318d01aeeb47c8814eebfc0f13fa12fa9"}
Jul 28 20:45:29.812975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:29.812620    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:45:29.836534 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:29.836481    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb" podStartSLOduration=1.836465527 podStartE2EDuration="1.836465527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:45:29.832066724 +0000 UTC m=+1225.017171429" watchObservedRunningTime="2026-07-28 20:45:29.836465527 +0000 UTC m=+1225.021570233"
Jul 28 20:45:40.820108 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:45:40.820076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k5fb"
Jul 28 20:48:20.477986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:48:20.477949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:48:36.504602 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:48:36.504562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:48:38.614045 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:48:38.614003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:49:00.793860 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:49:00.793771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:49:02.094344 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:49:02.094304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:49:41.247102 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:49:41.247062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:49:52.148028 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:49:52.147987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:07.870722 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:07.870683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:23.692726 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:23.692689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:47.916320 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:47.916286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:51.257908 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:51.257854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:51.351977 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:51.351944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:51.763244 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:51.763205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:58.303809 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:58.303771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:58.397559 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:58.397525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:58.540978 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:58.540947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:59.033901 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:59.033863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:52:59.328480 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:52:59.328400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:01.665831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.665800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:01.811185 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.811151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:01.861034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.861007    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:01.865129 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.865104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:01.873248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.873214    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2545433a-serving-cert\""
Jul 28 20:53:01.886594 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.886565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:01.934112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.934024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jp2s\" (UniqueName: \"kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:01.934112 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.934061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:01.934301 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.934149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:01.984488 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:01.984455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:02.034805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.034762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.035010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.034927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jp2s\" (UniqueName: \"kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.035010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.034963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.035542 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.035521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.037380 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.037346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.057274 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.057240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jp2s\" (UniqueName: \"kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s\") pod \"maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.142790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.142752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:02.176498 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.176461    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:02.530257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.530233    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:02.532089 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:53:02.532058    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90955308_7ea4_455c_b6d3_2627e296f7a6.slice/crio-98c9fbdeade7601ea0024eab6fa60469676181b0045ce179c3911d7e91271360 WatchSource:0}: Error finding container 98c9fbdeade7601ea0024eab6fa60469676181b0045ce179c3911d7e91271360: Status 404 returned error can't find the container with id 98c9fbdeade7601ea0024eab6fa60469676181b0045ce179c3911d7e91271360
Jul 28 20:53:02.533453 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.533436    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:53:02.662761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.662726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:02.763184 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.763142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:02.878683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.878577    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:02.883560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.883533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:02.887194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.887159    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2545433a\""
Jul 28 20:53:02.887546 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.887513    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2545433a-dockercfg-pthqv\""
Jul 28 20:53:02.896644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.896616    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:02.945021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.944977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:02.945203 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:02.945113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-882bn\" (UniqueName: \"kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.046074 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.046026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.046256 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.046134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-882bn\" (UniqueName: \"kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.047073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.047046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.064997 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.064961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-882bn\" (UniqueName: \"kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn\") pod \"payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.195690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.195650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:03.337287 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.337251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" event={"ID":"90955308-7ea4-455c-b6d3-2627e296f7a6","Type":"ContainerStarted","Data":"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"}
Jul 28 20:53:03.337518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.337313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" event={"ID":"90955308-7ea4-455c-b6d3-2627e296f7a6","Type":"ContainerStarted","Data":"98c9fbdeade7601ea0024eab6fa60469676181b0045ce179c3911d7e91271360"}
Jul 28 20:53:03.337518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.337380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:03.358351 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.358325    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:03.359061 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:53:03.359039    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb095668e_4766_4922_9da1_92428e164ec9.slice/crio-6f28e8c2daccdd374a44986fb533348bfc319233a2535138c2ad9459343cb232 WatchSource:0}: Error finding container 6f28e8c2daccdd374a44986fb533348bfc319233a2535138c2ad9459343cb232: Status 404 returned error can't find the container with id 6f28e8c2daccdd374a44986fb533348bfc319233a2535138c2ad9459343cb232
Jul 28 20:53:03.367472 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:03.367406    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" podStartSLOduration=1.744969501 podStartE2EDuration="2.367384001s" totalImagesPullingTime="622.4145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:03.365119838 +0000 UTC m=+1678.550224569" watchObservedRunningTime="2026-07-28 20:53:03.367384001 +0000 UTC m=+1678.552488708"
Jul 28 20:53:04.349547 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:04.349502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" event={"ID":"b095668e-4766-4922-9da1-92428e164ec9","Type":"ContainerStarted","Data":"6f28e8c2daccdd374a44986fb533348bfc319233a2535138c2ad9459343cb232"}
Jul 28 20:53:05.356755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:05.356718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" event={"ID":"b095668e-4766-4922-9da1-92428e164ec9","Type":"ContainerStarted","Data":"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"}
Jul 28 20:53:05.357118 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:05.356894    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:05.381115 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:05.381069    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" podStartSLOduration=1.7817890699999999 podStartE2EDuration="3.381052704s" totalImagesPullingTime="1.599263634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:05.379665052 +0000 UTC m=+1680.564769760" watchObservedRunningTime="2026-07-28 20:53:05.381052704 +0000 UTC m=+1680.566157425"
Jul 28 20:53:09.354971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:09.354937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:16.365241 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:16.365206    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:27.215784 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.215701    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"]
Jul 28 20:53:27.221300 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.221280    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:27.224151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.224118    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:53:27.227970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.227941    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"]
Jul 28 20:53:27.279122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.279086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2545433a-qxzn6\" (UID: \"fcf43e2f-105a-4f12-b9be-3ab022d0c321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:27.380400 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.380352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2545433a-qxzn6\" (UID: \"fcf43e2f-105a-4f12-b9be-3ab022d0c321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:27.381009 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.380985    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2545433a-qxzn6\" (UID: \"fcf43e2f-105a-4f12-b9be-3ab022d0c321\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:27.534225 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.534188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:27.679996 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:27.679927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"]
Jul 28 20:53:28.488996 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:28.488957    2576 generic.go:356] "Generic (PLEG): container finished" podID="fcf43e2f-105a-4f12-b9be-3ab022d0c321" containerID="bb46e30f3fd37b643563451e6f94ac8a451e4050d96213496d8a45d5c465a647" exitCode=0
Jul 28 20:53:28.489439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:28.489047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6" event={"ID":"fcf43e2f-105a-4f12-b9be-3ab022d0c321","Type":"ContainerDied","Data":"bb46e30f3fd37b643563451e6f94ac8a451e4050d96213496d8a45d5c465a647"}
Jul 28 20:53:28.489439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:28.489114    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6" event={"ID":"fcf43e2f-105a-4f12-b9be-3ab022d0c321","Type":"ContainerStarted","Data":"7d9b0d7d44617e912f63558a67076464026072657772025a6e5b836103a76ecd"}
Jul 28 20:53:29.635621 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:29.635590    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:29.702119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:29.702082    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca\") pod \"fcf43e2f-105a-4f12-b9be-3ab022d0c321\" (UID: \"fcf43e2f-105a-4f12-b9be-3ab022d0c321\") "
Jul 28 20:53:29.702468 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:29.702443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fcf43e2f-105a-4f12-b9be-3ab022d0c321" (UID: "fcf43e2f-105a-4f12-b9be-3ab022d0c321"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:29.803270 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:29.803185    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcf43e2f-105a-4f12-b9be-3ab022d0c321-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:30.501439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:30.501408    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"
Jul 28 20:53:30.501439 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:30.501424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6" event={"ID":"fcf43e2f-105a-4f12-b9be-3ab022d0c321","Type":"ContainerDied","Data":"7d9b0d7d44617e912f63558a67076464026072657772025a6e5b836103a76ecd"}
Jul 28 20:53:30.501687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:30.501471    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d9b0d7d44617e912f63558a67076464026072657772025a6e5b836103a76ecd"
Jul 28 20:53:32.421729 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.421695    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:32.422499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.422465    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" podUID="90955308-7ea4-455c-b6d3-2627e296f7a6" containerName="maas-api" containerID="cri-o://358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441" gracePeriod=30
Jul 28 20:53:32.453971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.453923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:32.494053 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.494018    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:32.494283 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.494251    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" podUID="b095668e-4766-4922-9da1-92428e164ec9" containerName="payload-pre-processing" containerID="cri-o://c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6" gracePeriod=30
Jul 28 20:53:32.649104 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.649069    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"]
Jul 28 20:53:32.651599 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.651561    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2545433a-qxzn6"]
Jul 28 20:53:32.757281 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.757254    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:32.782680 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.782651    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:32.828722 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.828686    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls\") pod \"90955308-7ea4-455c-b6d3-2627e296f7a6\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") "
Jul 28 20:53:32.828910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.828734    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-882bn\" (UniqueName: \"kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn\") pod \"b095668e-4766-4922-9da1-92428e164ec9\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") "
Jul 28 20:53:32.828910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.828802    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca\") pod \"90955308-7ea4-455c-b6d3-2627e296f7a6\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") "
Jul 28 20:53:32.828910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.828838    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jp2s\" (UniqueName: \"kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s\") pod \"90955308-7ea4-455c-b6d3-2627e296f7a6\" (UID: \"90955308-7ea4-455c-b6d3-2627e296f7a6\") "
Jul 28 20:53:32.828910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.828861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume\") pod \"b095668e-4766-4922-9da1-92428e164ec9\" (UID: \"b095668e-4766-4922-9da1-92428e164ec9\") "
Jul 28 20:53:32.829247 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.829219    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "90955308-7ea4-455c-b6d3-2627e296f7a6" (UID: "90955308-7ea4-455c-b6d3-2627e296f7a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:32.829359 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.829333    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b095668e-4766-4922-9da1-92428e164ec9" (UID: "b095668e-4766-4922-9da1-92428e164ec9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:53:32.830970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.830938    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn" (OuterVolumeSpecName: "kube-api-access-882bn") pod "b095668e-4766-4922-9da1-92428e164ec9" (UID: "b095668e-4766-4922-9da1-92428e164ec9"). InnerVolumeSpecName "kube-api-access-882bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:32.831072 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.831018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s" (OuterVolumeSpecName: "kube-api-access-6jp2s") pod "90955308-7ea4-455c-b6d3-2627e296f7a6" (UID: "90955308-7ea4-455c-b6d3-2627e296f7a6"). InnerVolumeSpecName "kube-api-access-6jp2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:32.831072 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.831048    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "90955308-7ea4-455c-b6d3-2627e296f7a6" (UID: "90955308-7ea4-455c-b6d3-2627e296f7a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:53:32.929616 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.929577    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90955308-7ea4-455c-b6d3-2627e296f7a6-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:32.929616 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.929609    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-882bn\" (UniqueName: \"kubernetes.io/projected/b095668e-4766-4922-9da1-92428e164ec9-kube-api-access-882bn\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:32.929616 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.929624    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90955308-7ea4-455c-b6d3-2627e296f7a6-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:32.929871 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.929635    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jp2s\" (UniqueName: \"kubernetes.io/projected/90955308-7ea4-455c-b6d3-2627e296f7a6-kube-api-access-6jp2s\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:32.929871 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:32.929649    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b095668e-4766-4922-9da1-92428e164ec9-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:53:33.301549 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.301515    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf43e2f-105a-4f12-b9be-3ab022d0c321" path="/var/lib/kubelet/pods/fcf43e2f-105a-4f12-b9be-3ab022d0c321/volumes"
Jul 28 20:53:33.524302 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.524263    2576 generic.go:356] "Generic (PLEG): container finished" podID="90955308-7ea4-455c-b6d3-2627e296f7a6" containerID="358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441" exitCode=0
Jul 28 20:53:33.524761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.524311    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"
Jul 28 20:53:33.524761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.524340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" event={"ID":"90955308-7ea4-455c-b6d3-2627e296f7a6","Type":"ContainerDied","Data":"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"}
Jul 28 20:53:33.524761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.524405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c" event={"ID":"90955308-7ea4-455c-b6d3-2627e296f7a6","Type":"ContainerDied","Data":"98c9fbdeade7601ea0024eab6fa60469676181b0045ce179c3911d7e91271360"}
Jul 28 20:53:33.524761 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.524439    2576 scope.go:117] "RemoveContainer" containerID="358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"
Jul 28 20:53:33.525733 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.525682    2576 generic.go:356] "Generic (PLEG): container finished" podID="b095668e-4766-4922-9da1-92428e164ec9" containerID="c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6" exitCode=0
Jul 28 20:53:33.525733 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.525716    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"
Jul 28 20:53:33.525860 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.525751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" event={"ID":"b095668e-4766-4922-9da1-92428e164ec9","Type":"ContainerDied","Data":"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"}
Jul 28 20:53:33.525860 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.525796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967" event={"ID":"b095668e-4766-4922-9da1-92428e164ec9","Type":"ContainerDied","Data":"6f28e8c2daccdd374a44986fb533348bfc319233a2535138c2ad9459343cb232"}
Jul 28 20:53:33.534253 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.534234    2576 scope.go:117] "RemoveContainer" containerID="358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"
Jul 28 20:53:33.534530 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:53:33.534511    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441\": container with ID starting with 358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441 not found: ID does not exist" containerID="358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"
Jul 28 20:53:33.534594 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.534540    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441"} err="failed to get container status \"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441\": rpc error: code = NotFound desc = could not find container \"358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441\": container with ID starting with 358ae2b34a72c29d85986905b6ef49c96a985145ce7c347b2db80001858f5441 not found: ID does not exist"
Jul 28 20:53:33.534594 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.534557    2576 scope.go:117] "RemoveContainer" containerID="c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"
Jul 28 20:53:33.544586 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.544568    2576 scope.go:117] "RemoveContainer" containerID="c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"
Jul 28 20:53:33.544889 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:53:33.544867    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6\": container with ID starting with c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6 not found: ID does not exist" containerID="c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"
Jul 28 20:53:33.544975 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.544900    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6"} err="failed to get container status \"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6\": rpc error: code = NotFound desc = could not find container \"c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6\": container with ID starting with c56bb12fe8ff35ccc1e54600f9aab333f239184434278642c591728994b116d6 not found: ID does not exist"
Jul 28 20:53:33.550939 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.550917    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:33.558260 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.558193    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2545433a-7685788dd4-9n967"]
Jul 28 20:53:33.574486 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.574454    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:33.579594 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:33.579567    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2545433a-dd66dddf9-xfh2c"]
Jul 28 20:53:34.469277 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:34.469242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:35.300856 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:35.300822    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90955308-7ea4-455c-b6d3-2627e296f7a6" path="/var/lib/kubelet/pods/90955308-7ea4-455c-b6d3-2627e296f7a6/volumes"
Jul 28 20:53:35.301238 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:35.301173    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b095668e-4766-4922-9da1-92428e164ec9" path="/var/lib/kubelet/pods/b095668e-4766-4922-9da1-92428e164ec9/volumes"
Jul 28 20:53:39.196962 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.196929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:39.302615 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.302580    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:53:39.302986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.302958    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf43e2f-105a-4f12-b9be-3ab022d0c321" containerName="revoke-keys"
Jul 28 20:53:39.302986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.302976    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf43e2f-105a-4f12-b9be-3ab022d0c321" containerName="revoke-keys"
Jul 28 20:53:39.302986 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.302991    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90955308-7ea4-455c-b6d3-2627e296f7a6" containerName="maas-api"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.302997    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90955308-7ea4-455c-b6d3-2627e296f7a6" containerName="maas-api"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.303011    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b095668e-4766-4922-9da1-92428e164ec9" containerName="payload-pre-processing"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.303018    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b095668e-4766-4922-9da1-92428e164ec9" containerName="payload-pre-processing"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.303078    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90955308-7ea4-455c-b6d3-2627e296f7a6" containerName="maas-api"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.303084    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf43e2f-105a-4f12-b9be-3ab022d0c321" containerName="revoke-keys"
Jul 28 20:53:39.303257 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.303092    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b095668e-4766-4922-9da1-92428e164ec9" containerName="payload-pre-processing"
Jul 28 20:53:39.306260 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.306244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.308743 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.308718    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e4b24854-serving-cert\""
Jul 28 20:53:39.314923 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.314873    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:53:39.337420 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.337393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:39.392564 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.392525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.392734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.392595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggfn\" (UniqueName: \"kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.392734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.392677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.458397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.458273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:39.493497 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.493455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.493628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.493539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggfn\" (UniqueName: \"kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.493628 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.493562    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.494152 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.494133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.496048 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.496030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.501909 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.501883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggfn\" (UniqueName: \"kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn\") pod \"maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.620790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.620757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:39.774887 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.774863    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:53:39.895704 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.895661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:39.988687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:39.988602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:53:40.569859 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:40.569827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" event={"ID":"69307842-6929-4bed-bcb3-4aedd93c008e","Type":"ContainerStarted","Data":"3eabb1b43732277dcad47112c0fccff3b4c2047a27b96f0dd1e02a430c3db3c7"}
Jul 28 20:53:42.582336 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:42.582282    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" event={"ID":"69307842-6929-4bed-bcb3-4aedd93c008e","Type":"ContainerStarted","Data":"2cfae4a49d58cb064be549a998e5b7301f508b00b5fcad5f243dd3b2c3a449d2"}
Jul 28 20:53:42.582767 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:42.582388    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:53:42.602838 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:42.602783    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" podStartSLOduration=1.685578238 podStartE2EDuration="3.602767778s" totalImagesPullingTime="1.91718954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:42.601612795 +0000 UTC m=+1717.786717503" watchObservedRunningTime="2026-07-28 20:53:42.602767778 +0000 UTC m=+1717.787872483"
Jul 28 20:53:48.593327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:53:48.593296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:54:04.303254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.303217    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"]
Jul 28 20:54:04.307105 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.307073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:04.309630 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.309601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:54:04.314354 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.314317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"]
Jul 28 20:54:04.419511 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.419471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e4b24854-7992k\" (UID: \"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:04.520801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.520761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e4b24854-7992k\" (UID: \"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:04.521454 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.521434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e4b24854-7992k\" (UID: \"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:04.619409 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.619298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:04.787886 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:04.787852    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"]
Jul 28 20:54:04.789621 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:04.789578    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fbea70_3b89_4299_a3a0_65b5ba3cf7da.slice/crio-303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01 WatchSource:0}: Error finding container 303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01: Status 404 returned error can't find the container with id 303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01
Jul 28 20:54:05.717970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:05.717872    2576 generic.go:356] "Generic (PLEG): container finished" podID="b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" containerID="30f4d951acedac717a969a802363b2393da326b30ee48cfd849c859d182cc455" exitCode=0
Jul 28 20:54:05.717970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:05.717929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k" event={"ID":"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da","Type":"ContainerDied","Data":"30f4d951acedac717a969a802363b2393da326b30ee48cfd849c859d182cc455"}
Jul 28 20:54:05.717970 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:05.717967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k" event={"ID":"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da","Type":"ContainerStarted","Data":"303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01"}
Jul 28 20:54:06.859319 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:06.859291    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:07.044309 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.044272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca\") pod \"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da\" (UID: \"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da\") "
Jul 28 20:54:07.044663 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.044639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" (UID: "b4fbea70-3b89-4299-a3a0-65b5ba3cf7da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:07.145806 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.145765    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:07.729755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.729674    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"
Jul 28 20:54:07.729755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.729707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k" event={"ID":"b4fbea70-3b89-4299-a3a0-65b5ba3cf7da","Type":"ContainerDied","Data":"303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01"}
Jul 28 20:54:07.729755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:07.729757    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="303d5ebe6937dce0ea9fcd9236fc55248cec69fd50884f0772dc01e1bb3b7f01"
Jul 28 20:54:10.618704 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.618664    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:54:10.619136 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.618948    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" podUID="69307842-6929-4bed-bcb3-4aedd93c008e" containerName="maas-api" containerID="cri-o://2cfae4a49d58cb064be549a998e5b7301f508b00b5fcad5f243dd3b2c3a449d2" gracePeriod=30
Jul 28 20:54:10.655750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.655719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:10.750990 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.750952    2576 generic.go:356] "Generic (PLEG): container finished" podID="69307842-6929-4bed-bcb3-4aedd93c008e" containerID="2cfae4a49d58cb064be549a998e5b7301f508b00b5fcad5f243dd3b2c3a449d2" exitCode=0
Jul 28 20:54:10.751171 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.751022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" event={"ID":"69307842-6929-4bed-bcb3-4aedd93c008e","Type":"ContainerDied","Data":"2cfae4a49d58cb064be549a998e5b7301f508b00b5fcad5f243dd3b2c3a449d2"}
Jul 28 20:54:10.757926 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.757898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:10.856253 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.856223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:10.875826 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.875766    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:54:10.876718 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.876699    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls\") pod \"69307842-6929-4bed-bcb3-4aedd93c008e\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") "
Jul 28 20:54:10.876803 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.876791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fggfn\" (UniqueName: \"kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn\") pod \"69307842-6929-4bed-bcb3-4aedd93c008e\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") "
Jul 28 20:54:10.876854 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.876826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca\") pod \"69307842-6929-4bed-bcb3-4aedd93c008e\" (UID: \"69307842-6929-4bed-bcb3-4aedd93c008e\") "
Jul 28 20:54:10.877202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.877181    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69307842-6929-4bed-bcb3-4aedd93c008e" (UID: "69307842-6929-4bed-bcb3-4aedd93c008e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:10.878971 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.878949    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn" (OuterVolumeSpecName: "kube-api-access-fggfn") pod "69307842-6929-4bed-bcb3-4aedd93c008e" (UID: "69307842-6929-4bed-bcb3-4aedd93c008e"). InnerVolumeSpecName "kube-api-access-fggfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:10.879068 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.879044    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69307842-6929-4bed-bcb3-4aedd93c008e" (UID: "69307842-6929-4bed-bcb3-4aedd93c008e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:54:10.882666 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.882637    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"]
Jul 28 20:54:10.885146 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.885125    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e4b24854-7992k"]
Jul 28 20:54:10.937910 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.937880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:10.977451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.977416    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggfn\" (UniqueName: \"kubernetes.io/projected/69307842-6929-4bed-bcb3-4aedd93c008e-kube-api-access-fggfn\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:10.977451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.977444    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69307842-6929-4bed-bcb3-4aedd93c008e-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:10.977451 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:10.977456    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69307842-6929-4bed-bcb3-4aedd93c008e-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:11.300788 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.300753    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" path="/var/lib/kubelet/pods/b4fbea70-3b89-4299-a3a0-65b5ba3cf7da/volumes"
Jul 28 20:54:11.758832 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.758798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r" event={"ID":"69307842-6929-4bed-bcb3-4aedd93c008e","Type":"ContainerDied","Data":"3eabb1b43732277dcad47112c0fccff3b4c2047a27b96f0dd1e02a430c3db3c7"}
Jul 28 20:54:11.759348 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.758875    2576 scope.go:117] "RemoveContainer" containerID="2cfae4a49d58cb064be549a998e5b7301f508b00b5fcad5f243dd3b2c3a449d2"
Jul 28 20:54:11.759348 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.758804    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"
Jul 28 20:54:11.767289 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.767259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:11.782175 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.782151    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:54:11.794908 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.794882    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e4b24854-6b74c7df55-s8k9r"]
Jul 28 20:54:11.870179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:11.870150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:12.007742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:12.007707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:12.439246 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:12.439210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:13.301163 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:13.301132    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69307842-6929-4bed-bcb3-4aedd93c008e" path="/var/lib/kubelet/pods/69307842-6929-4bed-bcb3-4aedd93c008e/volumes"
Jul 28 20:54:14.055798 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.055765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:14.144235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.144199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:14.221232 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221198    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:14.221626 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221611    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" containerName="revoke-keys"
Jul 28 20:54:14.221671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221627    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" containerName="revoke-keys"
Jul 28 20:54:14.221671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221646    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69307842-6929-4bed-bcb3-4aedd93c008e" containerName="maas-api"
Jul 28 20:54:14.221671 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221652    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="69307842-6929-4bed-bcb3-4aedd93c008e" containerName="maas-api"
Jul 28 20:54:14.221772 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221708    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="69307842-6929-4bed-bcb3-4aedd93c008e" containerName="maas-api"
Jul 28 20:54:14.221772 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.221718    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4fbea70-3b89-4299-a3a0-65b5ba3cf7da" containerName="revoke-keys"
Jul 28 20:54:14.225964 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.225943    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.228668 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.228646    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ee318a23-serving-cert\""
Jul 28 20:54:14.238505 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.238484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:14.251406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.251359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:14.307738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.307652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.307738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.307698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.308143 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.307777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn7pb\" (UniqueName: \"kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.409083 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.409042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.409278 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.409108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.409278 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.409185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn7pb\" (UniqueName: \"kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.409898 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.409865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.410160 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.410136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:14.411656 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.411639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.422166 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.422142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn7pb\" (UniqueName: \"kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb\") pod \"maas-api-e2e-ait-ee318a23-8469856458-98dcw\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.537099 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.537061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:14.690418 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.690344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:14.692480 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:14.692453    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5793f1_47d1_415c_84be_4159931f13bd.slice/crio-b0db355e43c434df82bb3193fb79346bf5732cce18feca3f4a36c5cf747c059b WatchSource:0}: Error finding container b0db355e43c434df82bb3193fb79346bf5732cce18feca3f4a36c5cf747c059b: Status 404 returned error can't find the container with id b0db355e43c434df82bb3193fb79346bf5732cce18feca3f4a36c5cf747c059b
Jul 28 20:54:14.739640 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.739608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:14.791486 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.791451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" event={"ID":"1f5793f1-47d1-415c-84be-4159931f13bd","Type":"ContainerStarted","Data":"b0db355e43c434df82bb3193fb79346bf5732cce18feca3f4a36c5cf747c059b"}
Jul 28 20:54:14.856093 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.856001    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:14.860982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.860960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:14.863694 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.863670    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ee318a23-dockercfg-rhpjn\""
Jul 28 20:54:14.863856 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.863839    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ee318a23\""
Jul 28 20:54:14.871017 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.870984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:14.876501 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.876478    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:14.880687 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.880659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:14.891694 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.891668    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:14.915016 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.914977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:14.915196 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.915054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6ldm\" (UniqueName: \"kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:14.915196 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.915129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnhsl\" (UniqueName: \"kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:14.915297 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:14.915195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.016119 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6ldm\" (UniqueName: \"kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.016312 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnhsl\" (UniqueName: \"kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.016312 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.016429 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.016952 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016933    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.016993 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.016980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.028077 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.028044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:15.028208 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.028166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6ldm\" (UniqueName: \"kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm\") pod \"payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.028249 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.028220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnhsl\" (UniqueName: \"kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl\") pod \"payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") " pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.174160 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.174135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.193447 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.193420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.329831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.329800    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:15.331083 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:15.330956    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831306c4_8e1d_48bf_a574_1f8c38e57ee8.slice/crio-9dc8486864df31571f019a22538ea7c0fa9a8eaa3994a9fa2d154d6c3efd109d WatchSource:0}: Error finding container 9dc8486864df31571f019a22538ea7c0fa9a8eaa3994a9fa2d154d6c3efd109d: Status 404 returned error can't find the container with id 9dc8486864df31571f019a22538ea7c0fa9a8eaa3994a9fa2d154d6c3efd109d
Jul 28 20:54:15.365667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.365634    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:15.367948 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:15.367887    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f364f7_f374_4ee7_b5ec_7abe2a2b4a1e.slice/crio-3a87103892f74dde09c1432002c93311746842ae8ac7c240fa5a6a319dc186cb WatchSource:0}: Error finding container 3a87103892f74dde09c1432002c93311746842ae8ac7c240fa5a6a319dc186cb: Status 404 returned error can't find the container with id 3a87103892f74dde09c1432002c93311746842ae8ac7c240fa5a6a319dc186cb
Jul 28 20:54:15.798648 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.798614    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" event={"ID":"1f5793f1-47d1-415c-84be-4159931f13bd","Type":"ContainerStarted","Data":"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"}
Jul 28 20:54:15.798867 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.798740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:15.800445 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.800417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" event={"ID":"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e","Type":"ContainerStarted","Data":"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"}
Jul 28 20:54:15.800577 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.800467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" event={"ID":"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e","Type":"ContainerStarted","Data":"3a87103892f74dde09c1432002c93311746842ae8ac7c240fa5a6a319dc186cb"}
Jul 28 20:54:15.800577 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.800499    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:15.801891 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.801874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" event={"ID":"831306c4-8e1d-48bf-a574-1f8c38e57ee8","Type":"ContainerStarted","Data":"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"}
Jul 28 20:54:15.801968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.801905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" event={"ID":"831306c4-8e1d-48bf-a574-1f8c38e57ee8","Type":"ContainerStarted","Data":"9dc8486864df31571f019a22538ea7c0fa9a8eaa3994a9fa2d154d6c3efd109d"}
Jul 28 20:54:15.802021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.802011    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:15.818339 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.818294    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" podStartSLOduration=1.394132109 podStartE2EDuration="1.818279962s" totalImagesPullingTime="424.147853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:15.816176616 +0000 UTC m=+1751.001281322" watchObservedRunningTime="2026-07-28 20:54:15.818279962 +0000 UTC m=+1751.003384668"
Jul 28 20:54:15.833917 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:15.833871    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" podStartSLOduration=1.833858013 podStartE2EDuration="1.833858013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:15.832543633 +0000 UTC m=+1751.017648350" watchObservedRunningTime="2026-07-28 20:54:15.833858013 +0000 UTC m=+1751.018962718"
Jul 28 20:54:21.812711 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:21.812683    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:21.830340 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:21.830287    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" podStartSLOduration=7.830272281 podStartE2EDuration="7.830272281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:15.852145384 +0000 UTC m=+1751.037250091" watchObservedRunningTime="2026-07-28 20:54:21.830272281 +0000 UTC m=+1757.015376988"
Jul 28 20:54:26.810232 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:26.810197    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:26.810645 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:26.810288    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:32.640021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:32.639988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:39.914341 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:39.914296    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"]
Jul 28 20:54:39.917787 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:39.917764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:39.920223 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:39.920205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:54:39.926560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:39.926533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"]
Jul 28 20:54:40.053264 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.053228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee318a23-64njh\" (UID: \"acf09e5d-7c0d-4655-b6a5-a8800e45eb85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:40.154273 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.154230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee318a23-64njh\" (UID: \"acf09e5d-7c0d-4655-b6a5-a8800e45eb85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:40.155066 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.155042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ee318a23-64njh\" (UID: \"acf09e5d-7c0d-4655-b6a5-a8800e45eb85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:40.231075 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.230976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:40.373385 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.373339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"]
Jul 28 20:54:40.374999 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:40.374956    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf09e5d_7c0d_4655_b6a5_a8800e45eb85.slice/crio-9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1 WatchSource:0}: Error finding container 9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1: Status 404 returned error can't find the container with id 9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1
Jul 28 20:54:40.950862 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.950775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh" event={"ID":"acf09e5d-7c0d-4655-b6a5-a8800e45eb85","Type":"ContainerStarted","Data":"02ea024caa7e490f882b34731ef9d7d21c64a731fd85ec8a505af3d695ee597d"}
Jul 28 20:54:40.950862 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.950843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh" event={"ID":"acf09e5d-7c0d-4655-b6a5-a8800e45eb85","Type":"ContainerStarted","Data":"9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1"}
Jul 28 20:54:40.966673 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:40.966625    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh" podStartSLOduration=1.9666083890000001 podStartE2EDuration="1.966608389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:40.966308784 +0000 UTC m=+1776.151413488" watchObservedRunningTime="2026-07-28 20:54:40.966608389 +0000 UTC m=+1776.151713086"
Jul 28 20:54:41.957787 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:41.957750    2576 generic.go:356] "Generic (PLEG): container finished" podID="acf09e5d-7c0d-4655-b6a5-a8800e45eb85" containerID="02ea024caa7e490f882b34731ef9d7d21c64a731fd85ec8a505af3d695ee597d" exitCode=0
Jul 28 20:54:41.958194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:41.957840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh" event={"ID":"acf09e5d-7c0d-4655-b6a5-a8800e45eb85","Type":"ContainerDied","Data":"02ea024caa7e490f882b34731ef9d7d21c64a731fd85ec8a505af3d695ee597d"}
Jul 28 20:54:43.102117 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.102092    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:43.182548 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.182518    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca\") pod \"acf09e5d-7c0d-4655-b6a5-a8800e45eb85\" (UID: \"acf09e5d-7c0d-4655-b6a5-a8800e45eb85\") "
Jul 28 20:54:43.182906 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.182882    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "acf09e5d-7c0d-4655-b6a5-a8800e45eb85" (UID: "acf09e5d-7c0d-4655-b6a5-a8800e45eb85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:43.283521 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.283491    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf09e5d-7c0d-4655-b6a5-a8800e45eb85-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:43.972055 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.972027    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"
Jul 28 20:54:43.972201 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.972023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh" event={"ID":"acf09e5d-7c0d-4655-b6a5-a8800e45eb85","Type":"ContainerDied","Data":"9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1"}
Jul 28 20:54:43.972201 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:43.972154    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9210093168b09b16c65425239a3e4bafa0138500c01c33955f2ab840e65a26a1"
Jul 28 20:54:44.911385 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.910702    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:44.911385 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.910948    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" podUID="1f5793f1-47d1-415c-84be-4159931f13bd" containerName="maas-api" containerID="cri-o://6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb" gracePeriod=30
Jul 28 20:54:44.951358 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.951316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:44.987603 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.987568    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:44.987833 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.987807    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" podUID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" containerName="payload-processing" containerID="cri-o://a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a" gracePeriod=30
Jul 28 20:54:44.996588 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.996556    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:44.996816 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:44.996782    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" podUID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" containerName="payload-pre-processing" containerID="cri-o://d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99" gracePeriod=30
Jul 28 20:54:45.031523 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.031495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:45.102386 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.102328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:45.149096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.149056    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"]
Jul 28 20:54:45.154353 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.154318    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ee318a23-64njh"]
Jul 28 20:54:45.179794 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.179728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:45.292239 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.292213    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:45.295085 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.295063    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:45.301473 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.301438    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf09e5d-7c0d-4655-b6a5-a8800e45eb85" path="/var/lib/kubelet/pods/acf09e5d-7c0d-4655-b6a5-a8800e45eb85/volumes"
Jul 28 20:54:45.310597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.310575    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:45.403520 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403482    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnhsl\" (UniqueName: \"kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl\") pod \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") "
Jul 28 20:54:45.403734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403554    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6ldm\" (UniqueName: \"kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm\") pod \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") "
Jul 28 20:54:45.403734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403605    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume\") pod \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\" (UID: \"831306c4-8e1d-48bf-a574-1f8c38e57ee8\") "
Jul 28 20:54:45.403734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume\") pod \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\" (UID: \"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e\") "
Jul 28 20:54:45.403734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403669    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca\") pod \"1f5793f1-47d1-415c-84be-4159931f13bd\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") "
Jul 28 20:54:45.403734 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403729    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls\") pod \"1f5793f1-47d1-415c-84be-4159931f13bd\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") "
Jul 28 20:54:45.404064 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.403761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn7pb\" (UniqueName: \"kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb\") pod \"1f5793f1-47d1-415c-84be-4159931f13bd\" (UID: \"1f5793f1-47d1-415c-84be-4159931f13bd\") "
Jul 28 20:54:45.404064 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.404039    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" (UID: "d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:45.404182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.404058    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "831306c4-8e1d-48bf-a574-1f8c38e57ee8" (UID: "831306c4-8e1d-48bf-a574-1f8c38e57ee8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:45.404182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.404063    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f5793f1-47d1-415c-84be-4159931f13bd" (UID: "1f5793f1-47d1-415c-84be-4159931f13bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:54:45.406219 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.406194    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm" (OuterVolumeSpecName: "kube-api-access-w6ldm") pod "831306c4-8e1d-48bf-a574-1f8c38e57ee8" (UID: "831306c4-8e1d-48bf-a574-1f8c38e57ee8"). InnerVolumeSpecName "kube-api-access-w6ldm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:45.406433 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.406406    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl" (OuterVolumeSpecName: "kube-api-access-vnhsl") pod "d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" (UID: "d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e"). InnerVolumeSpecName "kube-api-access-vnhsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:45.406433 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.406415    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb" (OuterVolumeSpecName: "kube-api-access-rn7pb") pod "1f5793f1-47d1-415c-84be-4159931f13bd" (UID: "1f5793f1-47d1-415c-84be-4159931f13bd"). InnerVolumeSpecName "kube-api-access-rn7pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:54:45.406568 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.406473    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f5793f1-47d1-415c-84be-4159931f13bd" (UID: "1f5793f1-47d1-415c-84be-4159931f13bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:54:45.504789 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504757    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnhsl\" (UniqueName: \"kubernetes.io/projected/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-kube-api-access-vnhsl\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504789 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504786    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6ldm\" (UniqueName: \"kubernetes.io/projected/831306c4-8e1d-48bf-a574-1f8c38e57ee8-kube-api-access-w6ldm\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504798    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/831306c4-8e1d-48bf-a574-1f8c38e57ee8-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504807    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504816    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f5793f1-47d1-415c-84be-4159931f13bd-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504827    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f5793f1-47d1-415c-84be-4159931f13bd-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.504994 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.504835    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn7pb\" (UniqueName: \"kubernetes.io/projected/1f5793f1-47d1-415c-84be-4159931f13bd-kube-api-access-rn7pb\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:54:45.984801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.984764    2576 generic.go:356] "Generic (PLEG): container finished" podID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" containerID="d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99" exitCode=0
Jul 28 20:54:45.985248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.984810    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"
Jul 28 20:54:45.985248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.984845    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" event={"ID":"831306c4-8e1d-48bf-a574-1f8c38e57ee8","Type":"ContainerDied","Data":"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"}
Jul 28 20:54:45.985248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.984903    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc" event={"ID":"831306c4-8e1d-48bf-a574-1f8c38e57ee8","Type":"ContainerDied","Data":"9dc8486864df31571f019a22538ea7c0fa9a8eaa3994a9fa2d154d6c3efd109d"}
Jul 28 20:54:45.985248 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.984930    2576 scope.go:117] "RemoveContainer" containerID="d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"
Jul 28 20:54:45.986293 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.986270    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f5793f1-47d1-415c-84be-4159931f13bd" containerID="6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb" exitCode=0
Jul 28 20:54:45.986408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.986317    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"
Jul 28 20:54:45.986408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.986379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" event={"ID":"1f5793f1-47d1-415c-84be-4159931f13bd","Type":"ContainerDied","Data":"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"}
Jul 28 20:54:45.986536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.986430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw" event={"ID":"1f5793f1-47d1-415c-84be-4159931f13bd","Type":"ContainerDied","Data":"b0db355e43c434df82bb3193fb79346bf5732cce18feca3f4a36c5cf747c059b"}
Jul 28 20:54:45.988048 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.988021    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" containerID="a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a" exitCode=0
Jul 28 20:54:45.988226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.988059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" event={"ID":"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e","Type":"ContainerDied","Data":"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"}
Jul 28 20:54:45.988226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.988076    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"
Jul 28 20:54:45.988226 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.988088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f" event={"ID":"d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e","Type":"ContainerDied","Data":"3a87103892f74dde09c1432002c93311746842ae8ac7c240fa5a6a319dc186cb"}
Jul 28 20:54:45.999256 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:45.999166    2576 scope.go:117] "RemoveContainer" containerID="d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"
Jul 28 20:54:46.000084 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:54:46.000042    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99\": container with ID starting with d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99 not found: ID does not exist" containerID="d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"
Jul 28 20:54:46.000199 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.000094    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99"} err="failed to get container status \"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99\": rpc error: code = NotFound desc = could not find container \"d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99\": container with ID starting with d1e5a824ce4e00b3ef6afb4731b1d143c5029caca869365f4642fb907b4adf99 not found: ID does not exist"
Jul 28 20:54:46.000199 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.000117    2576 scope.go:117] "RemoveContainer" containerID="6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"
Jul 28 20:54:46.009298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.009280    2576 scope.go:117] "RemoveContainer" containerID="6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"
Jul 28 20:54:46.009558 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:54:46.009537    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb\": container with ID starting with 6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb not found: ID does not exist" containerID="6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"
Jul 28 20:54:46.009606 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.009565    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb"} err="failed to get container status \"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb\": rpc error: code = NotFound desc = could not find container \"6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb\": container with ID starting with 6cca83d163e35a69e5e046e05185fef79294c7f75463ff74e38ccc5cb37b1adb not found: ID does not exist"
Jul 28 20:54:46.009606 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.009589    2576 scope.go:117] "RemoveContainer" containerID="a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"
Jul 28 20:54:46.017755 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.017733    2576 scope.go:117] "RemoveContainer" containerID="a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"
Jul 28 20:54:46.018064 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:54:46.018015    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a\": container with ID starting with a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a not found: ID does not exist" containerID="a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"
Jul 28 20:54:46.018164 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.018098    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a"} err="failed to get container status \"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a\": rpc error: code = NotFound desc = could not find container \"a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a\": container with ID starting with a961023012c224b1825bdbd6d7466a98362dc6631e03381634d909ca30369a8a not found: ID does not exist"
Jul 28 20:54:46.019458 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.019438    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:46.021831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.021812    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee318a23-8469856458-98dcw"]
Jul 28 20:54:46.031938 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.031913    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:46.038124 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.038107    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ee318a23-78cbd97ddd-29cnc"]
Jul 28 20:54:46.048851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.048819    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:46.053851 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.053830    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ee318a23-7978dcfcd4-tx76f"]
Jul 28 20:54:46.462752 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.462713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:46.694456 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.694419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:46.928403 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:46.928353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:47.301425 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:47.301391    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f5793f1-47d1-415c-84be-4159931f13bd" path="/var/lib/kubelet/pods/1f5793f1-47d1-415c-84be-4159931f13bd/volumes"
Jul 28 20:54:47.301915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:47.301888    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" path="/var/lib/kubelet/pods/831306c4-8e1d-48bf-a574-1f8c38e57ee8/volumes"
Jul 28 20:54:47.302258 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:47.302239    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" path="/var/lib/kubelet/pods/d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e/volumes"
Jul 28 20:54:47.311622 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:47.311595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:47.652953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:47.652859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:49.390494 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.390454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:49.494645 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.494606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:49.516220 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516189    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:54:49.516612 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516598    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" containerName="payload-processing"
Jul 28 20:54:49.516662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516614    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" containerName="payload-processing"
Jul 28 20:54:49.516662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516628    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" containerName="payload-pre-processing"
Jul 28 20:54:49.516662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516636    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" containerName="payload-pre-processing"
Jul 28 20:54:49.516662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516648    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf09e5d-7c0d-4655-b6a5-a8800e45eb85" containerName="revoke-keys"
Jul 28 20:54:49.516662 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516654    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf09e5d-7c0d-4655-b6a5-a8800e45eb85" containerName="revoke-keys"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516665    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5793f1-47d1-415c-84be-4159931f13bd" containerName="maas-api"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516671    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5793f1-47d1-415c-84be-4159931f13bd" containerName="maas-api"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516738    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="831306c4-8e1d-48bf-a574-1f8c38e57ee8" containerName="payload-pre-processing"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516747    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf09e5d-7c0d-4655-b6a5-a8800e45eb85" containerName="revoke-keys"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516752    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5793f1-47d1-415c-84be-4159931f13bd" containerName="maas-api"
Jul 28 20:54:49.516818 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.516758    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f364f7-f374-4ee7-b5ec-7abe2a2b4a1e" containerName="payload-processing"
Jul 28 20:54:49.521280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.521260    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.528449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.528425    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4871db53-serving-cert\""
Jul 28 20:54:49.537788 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.537765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:54:49.587170 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.587134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:49.654426 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.654333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.654565 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.654442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.654565 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.654523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk9zq\" (UniqueName: \"kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.756031 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.755996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.756188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.756079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk9zq\" (UniqueName: \"kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.756188 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:54:49.756100    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-serving-cert: secret "maas-api-e2e-derive-4871db53-serving-cert" not found
Jul 28 20:54:49.756188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.756118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.756188 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:54:49.756170    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls podName:775ed2ee-cd1a-4210-8cf1-bc6a5240e725 nodeName:}" failed. No retries permitted until 2026-07-28 20:54:50.25615231 +0000 UTC m=+1785.441256993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls") pod "maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" (UID: "775ed2ee-cd1a-4210-8cf1-bc6a5240e725") : secret "maas-api-e2e-derive-4871db53-serving-cert" not found
Jul 28 20:54:49.756738 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.756721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.767536 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.767514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk9zq\" (UniqueName: \"kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:49.814173 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:49.814146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:50.217272 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.217237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:50.261834 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.261791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:50.264240 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.264208    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") pod \"maas-api-e2e-derive-4871db53-56dd5d6f87-srdft\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:50.314100 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.314071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:54:50.367585 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.367547    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:54:50.371303 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.371287    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.373690 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.373667    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4871db53-dockercfg-8psz4\""
Jul 28 20:54:50.373974 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.373952    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4871db53\""
Jul 28 20:54:50.380674 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.380650    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:54:50.431167 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.431121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:50.463323 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.463292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.463499 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.463339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8b9\" (UniqueName: \"kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.490108 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.490060    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:54:50.495397 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.495335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.510990 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.510944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:54:50.564388 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.564141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.564388 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.564203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8b9\" (UniqueName: \"kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.565357 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.565327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.575669 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.575645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8b9\" (UniqueName: \"kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9\") pod \"payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.593797 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.593757    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:54:50.596132 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:50.596105    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775ed2ee_cd1a_4210_8cf1_bc6a5240e725.slice/crio-7ba67a0df4369d6de3fcd82ce53e920eab04f96477455bccc9cac95b76554939 WatchSource:0}: Error finding container 7ba67a0df4369d6de3fcd82ce53e920eab04f96477455bccc9cac95b76554939: Status 404 returned error can't find the container with id 7ba67a0df4369d6de3fcd82ce53e920eab04f96477455bccc9cac95b76554939
Jul 28 20:54:50.664888 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.664854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hg6m\" (UniqueName: \"kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.665069 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.664906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.684472 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.684436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:50.765983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.765918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hg6m\" (UniqueName: \"kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.766155 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.766044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.766782 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.766758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.773805 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.773782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hg6m\" (UniqueName: \"kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m\") pod \"payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") " pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.817966 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.817923    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:50.831560 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.831537    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:54:50.832160 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:50.832131    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4cbb46_1f5a_434a_b7cb_c078511942ec.slice/crio-aa668d3cf2ee6ba4274c69869dad16effb0c33e74b91e1827b7c24caf0667fd4 WatchSource:0}: Error finding container aa668d3cf2ee6ba4274c69869dad16effb0c33e74b91e1827b7c24caf0667fd4: Status 404 returned error can't find the container with id aa668d3cf2ee6ba4274c69869dad16effb0c33e74b91e1827b7c24caf0667fd4
Jul 28 20:54:50.966179 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:54:50.966150    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea99ee6_1944_4ad6_b910_d3b5ee3e9478.slice/crio-01854b7dd9928c52a05c5e13bba899db665e3fd0106307a7c932854c0f0a17c0 WatchSource:0}: Error finding container 01854b7dd9928c52a05c5e13bba899db665e3fd0106307a7c932854c0f0a17c0: Status 404 returned error can't find the container with id 01854b7dd9928c52a05c5e13bba899db665e3fd0106307a7c932854c0f0a17c0
Jul 28 20:54:50.967035 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:50.967008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:54:51.022807 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.022782    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" event={"ID":"eea99ee6-1944-4ad6-b910-d3b5ee3e9478","Type":"ContainerStarted","Data":"01854b7dd9928c52a05c5e13bba899db665e3fd0106307a7c932854c0f0a17c0"}
Jul 28 20:54:51.024496 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.024396    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" event={"ID":"7b4cbb46-1f5a-434a-b7cb-c078511942ec","Type":"ContainerStarted","Data":"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"}
Jul 28 20:54:51.024496 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.024457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" event={"ID":"7b4cbb46-1f5a-434a-b7cb-c078511942ec","Type":"ContainerStarted","Data":"aa668d3cf2ee6ba4274c69869dad16effb0c33e74b91e1827b7c24caf0667fd4"}
Jul 28 20:54:51.024742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.024710    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:54:51.025801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.025772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" event={"ID":"775ed2ee-cd1a-4210-8cf1-bc6a5240e725","Type":"ContainerStarted","Data":"7ba67a0df4369d6de3fcd82ce53e920eab04f96477455bccc9cac95b76554939"}
Jul 28 20:54:51.047667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:51.047610    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" podStartSLOduration=1.047591747 podStartE2EDuration="1.047591747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:51.042517366 +0000 UTC m=+1786.227622072" watchObservedRunningTime="2026-07-28 20:54:51.047591747 +0000 UTC m=+1786.232696454"
Jul 28 20:54:52.033223 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.033179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" event={"ID":"775ed2ee-cd1a-4210-8cf1-bc6a5240e725","Type":"ContainerStarted","Data":"b7e9bf2061a388a5b19091eb1cde495225e5e710d20cbb4e45725bd0512d9ea4"}
Jul 28 20:54:52.033685 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.033252    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:54:52.034846 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.034822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" event={"ID":"eea99ee6-1944-4ad6-b910-d3b5ee3e9478","Type":"ContainerStarted","Data":"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"}
Jul 28 20:54:52.034977 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.034864    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:54:52.057408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.057322    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" podStartSLOduration=2.679701257 podStartE2EDuration="3.057307082s" totalImagesPullingTime="377.605825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:52.056275118 +0000 UTC m=+1787.241379826" watchObservedRunningTime="2026-07-28 20:54:52.057307082 +0000 UTC m=+1787.242411789"
Jul 28 20:54:52.080113 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:52.080062    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" podStartSLOduration=2.080049671 podStartE2EDuration="2.080049671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:52.078067265 +0000 UTC m=+1787.263171962" watchObservedRunningTime="2026-07-28 20:54:52.080049671 +0000 UTC m=+1787.265154371"
Jul 28 20:54:58.049043 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:54:58.048957    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:55:02.036783 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:02.036733    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.72:9004: connect: connection refused"
Jul 28 20:55:03.044537 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:03.044498    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:55:12.036449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:12.036416    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:55:30.141000 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.140961    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:55:30.141800 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.141765    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" podUID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" containerName="maas-api" containerID="cri-o://b7e9bf2061a388a5b19091eb1cde495225e5e710d20cbb4e45725bd0512d9ea4" gracePeriod=30
Jul 28 20:55:30.163401 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.163325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:30.202915 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.202882    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:55:30.203152 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.203105    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" podUID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" containerName="payload-processing" containerID="cri-o://971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346" gracePeriod=30
Jul 28 20:55:30.210223 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.210193    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:55:30.210449 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.210422    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerName="payload-pre-processing" containerID="cri-o://ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d" gracePeriod=30
Jul 28 20:55:30.245401 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.245343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:30.262847 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.262817    2576 generic.go:356] "Generic (PLEG): container finished" podID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" containerID="b7e9bf2061a388a5b19091eb1cde495225e5e710d20cbb4e45725bd0512d9ea4" exitCode=0
Jul 28 20:55:30.262979 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.262889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" event={"ID":"775ed2ee-cd1a-4210-8cf1-bc6a5240e725","Type":"ContainerDied","Data":"b7e9bf2061a388a5b19091eb1cde495225e5e710d20cbb4e45725bd0512d9ea4"}
Jul 28 20:55:30.312983 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.312953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:30.414455 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.414377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:30.478494 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.478085    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:55:30.504721 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.504694    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:55:30.529982 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.529951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:30.539398 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.533935    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:55:30.638866 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.638824    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk9zq\" (UniqueName: \"kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq\") pod \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") "
Jul 28 20:55:30.639059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.638883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca\") pod \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") "
Jul 28 20:55:30.639059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.638908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume\") pod \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") "
Jul 28 20:55:30.639059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.638960    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hg6m\" (UniqueName: \"kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m\") pod \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\" (UID: \"eea99ee6-1944-4ad6-b910-d3b5ee3e9478\") "
Jul 28 20:55:30.639059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.639010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8b9\" (UniqueName: \"kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9\") pod \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") "
Jul 28 20:55:30.639059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.639045    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume\") pod \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\" (UID: \"7b4cbb46-1f5a-434a-b7cb-c078511942ec\") "
Jul 28 20:55:30.639349 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.639087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") pod \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\" (UID: \"775ed2ee-cd1a-4210-8cf1-bc6a5240e725\") "
Jul 28 20:55:30.639837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.639803    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eea99ee6-1944-4ad6-b910-d3b5ee3e9478" (UID: "eea99ee6-1944-4ad6-b910-d3b5ee3e9478"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:55:30.640127 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.640014    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "775ed2ee-cd1a-4210-8cf1-bc6a5240e725" (UID: "775ed2ee-cd1a-4210-8cf1-bc6a5240e725"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:55:30.640236 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.639827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b4cbb46-1f5a-434a-b7cb-c078511942ec" (UID: "7b4cbb46-1f5a-434a-b7cb-c078511942ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:55:30.641956 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.641917    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9" (OuterVolumeSpecName: "kube-api-access-nd8b9") pod "7b4cbb46-1f5a-434a-b7cb-c078511942ec" (UID: "7b4cbb46-1f5a-434a-b7cb-c078511942ec"). InnerVolumeSpecName "kube-api-access-nd8b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:30.642206 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.642177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m" (OuterVolumeSpecName: "kube-api-access-7hg6m") pod "eea99ee6-1944-4ad6-b910-d3b5ee3e9478" (UID: "eea99ee6-1944-4ad6-b910-d3b5ee3e9478"). InnerVolumeSpecName "kube-api-access-7hg6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:30.642298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.642219    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq" (OuterVolumeSpecName: "kube-api-access-qk9zq") pod "775ed2ee-cd1a-4210-8cf1-bc6a5240e725" (UID: "775ed2ee-cd1a-4210-8cf1-bc6a5240e725"). InnerVolumeSpecName "kube-api-access-qk9zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:30.642298 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.642234    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "775ed2ee-cd1a-4210-8cf1-bc6a5240e725" (UID: "775ed2ee-cd1a-4210-8cf1-bc6a5240e725"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:55:30.739839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739807    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd8b9\" (UniqueName: \"kubernetes.io/projected/7b4cbb46-1f5a-434a-b7cb-c078511942ec-kube-api-access-nd8b9\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.739839 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739832    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b4cbb46-1f5a-434a-b7cb-c078511942ec-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.740021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739842    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.740021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739854    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk9zq\" (UniqueName: \"kubernetes.io/projected/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-kube-api-access-qk9zq\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.740021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739864    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ed2ee-cd1a-4210-8cf1-bc6a5240e725-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.740021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739873    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.740021 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.739882    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hg6m\" (UniqueName: \"kubernetes.io/projected/eea99ee6-1944-4ad6-b910-d3b5ee3e9478-kube-api-access-7hg6m\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:55:30.780301 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:30.780266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:31.271406 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.271350    2576 generic.go:356] "Generic (PLEG): container finished" podID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" containerID="971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346" exitCode=0
Jul 28 20:55:31.271852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.271446    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"
Jul 28 20:55:31.271852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.271446    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" event={"ID":"eea99ee6-1944-4ad6-b910-d3b5ee3e9478","Type":"ContainerDied","Data":"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"}
Jul 28 20:55:31.271852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.271603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw" event={"ID":"eea99ee6-1944-4ad6-b910-d3b5ee3e9478","Type":"ContainerDied","Data":"01854b7dd9928c52a05c5e13bba899db665e3fd0106307a7c932854c0f0a17c0"}
Jul 28 20:55:31.271852 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.271639    2576 scope.go:117] "RemoveContainer" containerID="971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"
Jul 28 20:55:31.273061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.272866    2576 generic.go:356] "Generic (PLEG): container finished" podID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerID="ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d" exitCode=0
Jul 28 20:55:31.273061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.272928    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"
Jul 28 20:55:31.273061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.272953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" event={"ID":"7b4cbb46-1f5a-434a-b7cb-c078511942ec","Type":"ContainerDied","Data":"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"}
Jul 28 20:55:31.273061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.273009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh" event={"ID":"7b4cbb46-1f5a-434a-b7cb-c078511942ec","Type":"ContainerDied","Data":"aa668d3cf2ee6ba4274c69869dad16effb0c33e74b91e1827b7c24caf0667fd4"}
Jul 28 20:55:31.274739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.274712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft" event={"ID":"775ed2ee-cd1a-4210-8cf1-bc6a5240e725","Type":"ContainerDied","Data":"7ba67a0df4369d6de3fcd82ce53e920eab04f96477455bccc9cac95b76554939"}
Jul 28 20:55:31.274739 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.274736    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"
Jul 28 20:55:31.282251 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.282224    2576 scope.go:117] "RemoveContainer" containerID="971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"
Jul 28 20:55:31.282742 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:55:31.282584    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346\": container with ID starting with 971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346 not found: ID does not exist" containerID="971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"
Jul 28 20:55:31.282742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.282618    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346"} err="failed to get container status \"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346\": rpc error: code = NotFound desc = could not find container \"971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346\": container with ID starting with 971b95187868964649afb1ed7d0fa17fd6ebf43d95351083ea91b3caf2ab5346 not found: ID does not exist"
Jul 28 20:55:31.282742 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.282641    2576 scope.go:117] "RemoveContainer" containerID="ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"
Jul 28 20:55:31.293981 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.293959    2576 scope.go:117] "RemoveContainer" containerID="ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"
Jul 28 20:55:31.294683 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:55:31.294496    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d\": container with ID starting with ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d not found: ID does not exist" containerID="ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"
Jul 28 20:55:31.294683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.294541    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d"} err="failed to get container status \"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d\": rpc error: code = NotFound desc = could not find container \"ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d\": container with ID starting with ff5050426eb333f4ece0792bd9b642ef06663faeb95c2d98f5b879795d85fa7d not found: ID does not exist"
Jul 28 20:55:31.294683 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.294566    2576 scope.go:117] "RemoveContainer" containerID="b7e9bf2061a388a5b19091eb1cde495225e5e710d20cbb4e45725bd0512d9ea4"
Jul 28 20:55:31.305418 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.305397    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:55:31.306341 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.306318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:31.313548 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.313527    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-4871db53-77c5597f7d-zmhnw"]
Jul 28 20:55:31.341521 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.341483    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:55:31.349209 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.349178    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4871db53-78d6f5f99b-6bbqh"]
Jul 28 20:55:31.362240 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.362207    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:55:31.364351 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.364329    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4871db53-56dd5d6f87-srdft"]
Jul 28 20:55:31.399037 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.399008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:31.503317 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:31.503290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:33.110393 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:33.110336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:33.300810 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:33.300768    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" path="/var/lib/kubelet/pods/775ed2ee-cd1a-4210-8cf1-bc6a5240e725/volumes"
Jul 28 20:55:33.301198 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:33.301177    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" path="/var/lib/kubelet/pods/7b4cbb46-1f5a-434a-b7cb-c078511942ec/volumes"
Jul 28 20:55:33.301590 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:33.301572    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" path="/var/lib/kubelet/pods/eea99ee6-1944-4ad6-b910-d3b5ee3e9478/volumes"
Jul 28 20:55:39.970870 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:39.970822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:40.085675 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:40.085636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:40.332833 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:40.332789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:40.487801 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:40.487757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:55:40.875972 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:55:40.875916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:06.973750 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:06.973715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:13.342535 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:13.342494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:13.444722 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:13.444664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:13.545022 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:13.544969    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:29.578151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:29.578070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:35.660595 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:35.660559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:35.772161 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:35.772122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:35.891202 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:35.891168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:46.453684 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:46.453647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:47.274558 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:47.274522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:53.894639 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:53.894604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:53.986991 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:53.986916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:54.077035 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:54.077002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:54.182336 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:54.182251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:59.449292 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:59.449253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:59.547194 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:59.547146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:56:59.646580 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:56:59.646539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:01.497015 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:01.496977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:07.898254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:07.898131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:08.008350 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:08.008297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:08.674299 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:08.674261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:14.709073 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:14.709038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:14.809375 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:14.809330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:14.987537 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:14.987454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:15.331265 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:15.331231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:15.835790 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:15.835751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:18.653199 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.653158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:18.742067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.742032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:18.820707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.820658    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:57:18.821085 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821072    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" containerName="maas-api"
Jul 28 20:57:18.821134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821087    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" containerName="maas-api"
Jul 28 20:57:18.821134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821107    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" containerName="payload-processing"
Jul 28 20:57:18.821134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821115    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" containerName="payload-processing"
Jul 28 20:57:18.821134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerName="payload-pre-processing"
Jul 28 20:57:18.821134 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821130    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerName="payload-pre-processing"
Jul 28 20:57:18.821296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821183    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4cbb46-1f5a-434a-b7cb-c078511942ec" containerName="payload-pre-processing"
Jul 28 20:57:18.821296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821193    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="775ed2ee-cd1a-4210-8cf1-bc6a5240e725" containerName="maas-api"
Jul 28 20:57:18.821296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.821199    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eea99ee6-1944-4ad6-b910-d3b5ee3e9478" containerName="payload-processing"
Jul 28 20:57:18.824237 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.824221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:18.826686 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.826664    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2c06c6bc-serving-cert\""
Jul 28 20:57:18.834828 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.834803    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:57:18.897833 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.897792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:18.920644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.920556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:18.920644 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.920635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:18.920837 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:18.920712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-956cx\" (UniqueName: \"kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.022109 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.022068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.022280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.022132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.022280 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.022208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-956cx\" (UniqueName: \"kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.022388 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:57:19.022304    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-serving-cert: secret "maas-api-e2e-mt-2c06c6bc-serving-cert" not found
Jul 28 20:57:19.022428 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:57:19.022407    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls podName:951f56f3-0879-4ed2-8c9b-0bf4d335243b nodeName:}" failed. No retries permitted until 2026-07-28 20:57:19.522357014 +0000 UTC m=+1934.707461718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls") pod "maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" (UID: "951f56f3-0879-4ed2-8c9b-0bf4d335243b") : secret "maas-api-e2e-mt-2c06c6bc-serving-cert" not found
Jul 28 20:57:19.022968 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.022944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.031225 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.031197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-956cx\" (UniqueName: \"kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.431390 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.431337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:19.525010 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.524977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:19.528296 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.528273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:19.528434 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:57:19.528418    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-serving-cert: secret "maas-api-e2e-mt-2c06c6bc-serving-cert" not found
Jul 28 20:57:19.528487 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:57:19.528478    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls podName:951f56f3-0879-4ed2-8c9b-0bf4d335243b nodeName:}" failed. No retries permitted until 2026-07-28 20:57:20.528463577 +0000 UTC m=+1935.713568264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls") pod "maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" (UID: "951f56f3-0879-4ed2-8c9b-0bf4d335243b") : secret "maas-api-e2e-mt-2c06c6bc-serving-cert" not found
Jul 28 20:57:19.632589 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:19.632544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:20.537691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:20.537648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:20.540659 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:20.540631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:20.636123 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:20.636080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:20.782122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:20.782084    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:57:20.784493 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:57:20.784466    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951f56f3_0879_4ed2_8c9b_0bf4d335243b.slice/crio-f9e4925327bc0f8faa7235ccf0ab73c174f098e1f426fc5a4401d4870c6e961c WatchSource:0}: Error finding container f9e4925327bc0f8faa7235ccf0ab73c174f098e1f426fc5a4401d4870c6e961c: Status 404 returned error can't find the container with id f9e4925327bc0f8faa7235ccf0ab73c174f098e1f426fc5a4401d4870c6e961c
Jul 28 20:57:20.930182 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:20.930147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" event={"ID":"951f56f3-0879-4ed2-8c9b-0bf4d335243b","Type":"ContainerStarted","Data":"f9e4925327bc0f8faa7235ccf0ab73c174f098e1f426fc5a4401d4870c6e961c"}
Jul 28 20:57:21.937313 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:21.937276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" event={"ID":"951f56f3-0879-4ed2-8c9b-0bf4d335243b","Type":"ContainerStarted","Data":"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"}
Jul 28 20:57:21.937716 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:21.937410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:21.958661 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:21.958609    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" podStartSLOduration=3.569876685 podStartE2EDuration="3.958592875s" totalImagesPullingTime="388.71619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:57:21.95676438 +0000 UTC m=+1937.141869086" watchObservedRunningTime="2026-07-28 20:57:21.958592875 +0000 UTC m=+1937.143697580"
Jul 28 20:57:27.950254 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:27.950219    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:57:41.220454 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:41.220417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:43.763723 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:43.763676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:44.484580 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:44.484531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:44.587506 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:44.587448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:44.948238 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:44.948202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:50.761100 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:50.761042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:50.827307 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:50.827267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:51.147191 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.147083    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"]
Jul 28 20:57:51.150658 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.150634    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:51.153123 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.153102    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:57:51.159892 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.158840    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"]
Jul 28 20:57:51.320959 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.320927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv\" (UID: \"2f210fe7-e879-46f4-8ccb-11be00b4e652\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:51.422216 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.422129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv\" (UID: \"2f210fe7-e879-46f4-8ccb-11be00b4e652\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:51.423229 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.422902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv\" (UID: \"2f210fe7-e879-46f4-8ccb-11be00b4e652\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:51.463111 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.463070    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:51.601771 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.601744    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"]
Jul 28 20:57:51.603414 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:57:51.603386    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f210fe7_e879_46f4_8ccb_11be00b4e652.slice/crio-047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd WatchSource:0}: Error finding container 047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd: Status 404 returned error can't find the container with id 047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd
Jul 28 20:57:51.724322 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:51.724236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:57:52.109110 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:52.109071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv" event={"ID":"2f210fe7-e879-46f4-8ccb-11be00b4e652","Type":"ContainerStarted","Data":"c8af830eade088a81d991f695d8fa7cdca301145612c5c9ef735799fe8db0e7c"}
Jul 28 20:57:52.109518 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:52.109142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv" event={"ID":"2f210fe7-e879-46f4-8ccb-11be00b4e652","Type":"ContainerStarted","Data":"047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd"}
Jul 28 20:57:52.127979 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:52.127923    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv" podStartSLOduration=1.127905154 podStartE2EDuration="1.127905154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:57:52.124541097 +0000 UTC m=+1967.309645804" watchObservedRunningTime="2026-07-28 20:57:52.127905154 +0000 UTC m=+1967.313009861"
Jul 28 20:57:53.116892 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:53.116859    2576 generic.go:356] "Generic (PLEG): container finished" podID="2f210fe7-e879-46f4-8ccb-11be00b4e652" containerID="c8af830eade088a81d991f695d8fa7cdca301145612c5c9ef735799fe8db0e7c" exitCode=0
Jul 28 20:57:53.117284 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:53.116957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv" event={"ID":"2f210fe7-e879-46f4-8ccb-11be00b4e652","Type":"ContainerDied","Data":"c8af830eade088a81d991f695d8fa7cdca301145612c5c9ef735799fe8db0e7c"}
Jul 28 20:57:54.257483 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:54.257456    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:54.353328 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:54.353289    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca\") pod \"2f210fe7-e879-46f4-8ccb-11be00b4e652\" (UID: \"2f210fe7-e879-46f4-8ccb-11be00b4e652\") "
Jul 28 20:57:54.353705 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:54.353674    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f210fe7-e879-46f4-8ccb-11be00b4e652" (UID: "2f210fe7-e879-46f4-8ccb-11be00b4e652"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:57:54.455123 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:54.455017    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f210fe7-e879-46f4-8ccb-11be00b4e652-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:57:55.129872 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:55.129843    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"
Jul 28 20:57:55.130076 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:55.129840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv" event={"ID":"2f210fe7-e879-46f4-8ccb-11be00b4e652","Type":"ContainerDied","Data":"047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd"}
Jul 28 20:57:55.130076 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:57:55.129974    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="047ec7e7a4cea0673a11a5331d4b25dcea98ceea43da51d3c8bd8e4867833dfd"
Jul 28 20:58:01.761710 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:01.761606    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:58:01.762691 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:01.761888    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" podUID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" containerName="maas-api" containerID="cri-o://da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823" gracePeriod=30
Jul 28 20:58:01.801532 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:01.801499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:01.929131 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:01.929047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:02.000215 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.000185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:02.043113 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.043083    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:58:02.050482 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.050446    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"]
Jul 28 20:58:02.053427 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.053402    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2c06c6bc-zb5tv"]
Jul 28 20:58:02.070101 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.070071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:02.122408 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.122343    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca\") pod \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") "
Jul 28 20:58:02.122564 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.122422    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") pod \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") "
Jul 28 20:58:02.122564 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.122502    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-956cx\" (UniqueName: \"kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx\") pod \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\" (UID: \"951f56f3-0879-4ed2-8c9b-0bf4d335243b\") "
Jul 28 20:58:02.122760 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.122737    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "951f56f3-0879-4ed2-8c9b-0bf4d335243b" (UID: "951f56f3-0879-4ed2-8c9b-0bf4d335243b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:58:02.124626 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.124595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "951f56f3-0879-4ed2-8c9b-0bf4d335243b" (UID: "951f56f3-0879-4ed2-8c9b-0bf4d335243b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:58:02.124752 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.124674    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx" (OuterVolumeSpecName: "kube-api-access-956cx") pod "951f56f3-0879-4ed2-8c9b-0bf4d335243b" (UID: "951f56f3-0879-4ed2-8c9b-0bf4d335243b"). InnerVolumeSpecName "kube-api-access-956cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:58:02.171001 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.170969    2576 generic.go:356] "Generic (PLEG): container finished" podID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" containerID="da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823" exitCode=0
Jul 28 20:58:02.171188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.171012    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"
Jul 28 20:58:02.171188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.171057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" event={"ID":"951f56f3-0879-4ed2-8c9b-0bf4d335243b","Type":"ContainerDied","Data":"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"}
Jul 28 20:58:02.171188 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.171162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh" event={"ID":"951f56f3-0879-4ed2-8c9b-0bf4d335243b","Type":"ContainerDied","Data":"f9e4925327bc0f8faa7235ccf0ab73c174f098e1f426fc5a4401d4870c6e961c"}
Jul 28 20:58:02.171308 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.171198    2576 scope.go:117] "RemoveContainer" containerID="da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"
Jul 28 20:58:02.183714 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.183690    2576 scope.go:117] "RemoveContainer" containerID="da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"
Jul 28 20:58:02.184078 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:58:02.184055    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823\": container with ID starting with da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823 not found: ID does not exist" containerID="da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"
Jul 28 20:58:02.184180 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.184090    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823"} err="failed to get container status \"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823\": rpc error: code = NotFound desc = could not find container \"da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823\": container with ID starting with da02c51adbff80cb8796523c6b748dcd4e73742da5ba77cf0b5dc858e1b92823 not found: ID does not exist"
Jul 28 20:58:02.196091 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.196054    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:58:02.201310 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.201279    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2c06c6bc-857c7f75d8-r5qgh"]
Jul 28 20:58:02.223179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.223143    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-956cx\" (UniqueName: \"kubernetes.io/projected/951f56f3-0879-4ed2-8c9b-0bf4d335243b-kube-api-access-956cx\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:58:02.223179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.223177    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951f56f3-0879-4ed2-8c9b-0bf4d335243b-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:58:02.223179 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.223188    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951f56f3-0879-4ed2-8c9b-0bf4d335243b-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:58:02.431992 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:02.431902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:03.300641 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:03.300610    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f210fe7-e879-46f4-8ccb-11be00b4e652" path="/var/lib/kubelet/pods/2f210fe7-e879-46f4-8ccb-11be00b4e652/volumes"
Jul 28 20:58:03.301027 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:03.300989    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" path="/var/lib/kubelet/pods/951f56f3-0879-4ed2-8c9b-0bf4d335243b/volumes"
Jul 28 20:58:08.866026 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:08.865994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:08.958577 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:08.958544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:09.250995 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:09.250967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:09.430120 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:09.430085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:09.549120 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:09.549040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:09.641622 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:09.641590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:09.788206 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:09.788172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:15.802567 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:15.802535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:15.865515 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:15.865474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:15.954489 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:15.954441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:16.046877 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:16.046845    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:16.181227 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:16.181139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:22.297267 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:22.297227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:22.385172 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:22.385142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:29.448460 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:29.448419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:29.762295 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:29.762260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:29.975106 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:29.975075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:32.996505 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:32.996454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:33.100437 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.100402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:33.107682 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.107651    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:58:33.108034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108020    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f210fe7-e879-46f4-8ccb-11be00b4e652" containerName="revoke-keys"
Jul 28 20:58:33.108034 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108034    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f210fe7-e879-46f4-8ccb-11be00b4e652" containerName="revoke-keys"
Jul 28 20:58:33.108128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108046    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" containerName="maas-api"
Jul 28 20:58:33.108128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108052    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" containerName="maas-api"
Jul 28 20:58:33.108128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108120    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="951f56f3-0879-4ed2-8c9b-0bf4d335243b" containerName="maas-api"
Jul 28 20:58:33.108128 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.108128    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f210fe7-e879-46f4-8ccb-11be00b4e652" containerName="revoke-keys"
Jul 28 20:58:33.111053 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.111034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.118824 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.118800    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a8387f77-serving-cert\""
Jul 28 20:58:33.124096 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.124072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:58:33.209235 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.209196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:33.209668 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.209648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.209722 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.209704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.209782 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.209763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdqh\" (UniqueName: \"kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.310653 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.310561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.310653 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.310607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdqh\" (UniqueName: \"kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.310863 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.310829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.311513 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.311492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.313054 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.313036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.326109 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.326078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdqh\" (UniqueName: \"kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh\") pod \"maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.365153 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.365115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:33.422709 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.422675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:33.573877 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.573845    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:58:33.575658 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:58:33.575631    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb576a578_b4d7_4db2_ad5d_19b03b6f79c5.slice/crio-1fb34d63881f346912873eabd96f945539eedbe6fcca01ecae7a11bb86200209 WatchSource:0}: Error finding container 1fb34d63881f346912873eabd96f945539eedbe6fcca01ecae7a11bb86200209: Status 404 returned error can't find the container with id 1fb34d63881f346912873eabd96f945539eedbe6fcca01ecae7a11bb86200209
Jul 28 20:58:33.577259 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:33.577240    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:58:34.123678 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.123639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:34.257555 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.257521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:34.360122 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.360089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" event={"ID":"b576a578-b4d7-4db2-ad5d-19b03b6f79c5","Type":"ContainerStarted","Data":"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"}
Jul 28 20:58:34.360306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.360144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" event={"ID":"b576a578-b4d7-4db2-ad5d-19b03b6f79c5","Type":"ContainerStarted","Data":"1fb34d63881f346912873eabd96f945539eedbe6fcca01ecae7a11bb86200209"}
Jul 28 20:58:34.360306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.360180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:34.385067 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:34.385011    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" podStartSLOduration=0.984631419 podStartE2EDuration="1.384994125s" totalImagesPullingTime="400.362706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:58:34.381862765 +0000 UTC m=+2009.566967471" watchObservedRunningTime="2026-07-28 20:58:34.384994125 +0000 UTC m=+2009.570098833"
Jul 28 20:58:40.370953 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:40.370923    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:58:51.933556 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:51.933498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:58:58.072707 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:58:58.072668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:08.931964 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:08.931927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:30.634844 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:30.634756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:30.694141 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:30.694104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:33.354476 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.354441    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"]
Jul 28 20:59:33.358047 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.358022    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:33.360600 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.360580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 20:59:33.365327 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.365300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"]
Jul 28 20:59:33.372398 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.372374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5\" (UID: \"56791520-4e95-4361-84c5-c48da97a1f7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:33.473502 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.473464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5\" (UID: \"56791520-4e95-4361-84c5-c48da97a1f7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:33.474107 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.474082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5\" (UID: \"56791520-4e95-4361-84c5-c48da97a1f7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:33.671086 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.670996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:33.816896 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:33.816864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"]
Jul 28 20:59:33.818089 ip-10-0-139-131 kubenswrapper[2576]: W0728 20:59:33.818059    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56791520_4e95_4361_84c5_c48da97a1f7a.slice/crio-52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c WatchSource:0}: Error finding container 52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c: Status 404 returned error can't find the container with id 52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c
Jul 28 20:59:34.702876 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:34.702841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5" event={"ID":"56791520-4e95-4361-84c5-c48da97a1f7a","Type":"ContainerStarted","Data":"5d5e573fff9796d6b73f3ddb5c94e0f10e2c17f4b6b3cc36a8929b004e29262a"}
Jul 28 20:59:34.703290 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:34.702896    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5" event={"ID":"56791520-4e95-4361-84c5-c48da97a1f7a","Type":"ContainerStarted","Data":"52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c"}
Jul 28 20:59:34.732567 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:34.732517    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5" podStartSLOduration=1.7325025680000001 podStartE2EDuration="1.732502568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:59:34.730712508 +0000 UTC m=+2069.915817213" watchObservedRunningTime="2026-07-28 20:59:34.732502568 +0000 UTC m=+2069.917607279"
Jul 28 20:59:35.709855 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:35.709818    2576 generic.go:356] "Generic (PLEG): container finished" podID="56791520-4e95-4361-84c5-c48da97a1f7a" containerID="5d5e573fff9796d6b73f3ddb5c94e0f10e2c17f4b6b3cc36a8929b004e29262a" exitCode=0
Jul 28 20:59:35.710259 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:35.709903    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5" event={"ID":"56791520-4e95-4361-84c5-c48da97a1f7a","Type":"ContainerDied","Data":"5d5e573fff9796d6b73f3ddb5c94e0f10e2c17f4b6b3cc36a8929b004e29262a"}
Jul 28 20:59:36.855831 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:36.855807    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:36.902508 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:36.902480    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca\") pod \"56791520-4e95-4361-84c5-c48da97a1f7a\" (UID: \"56791520-4e95-4361-84c5-c48da97a1f7a\") "
Jul 28 20:59:36.902814 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:36.902792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56791520-4e95-4361-84c5-c48da97a1f7a" (UID: "56791520-4e95-4361-84c5-c48da97a1f7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:59:37.003151 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:37.003118    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56791520-4e95-4361-84c5-c48da97a1f7a-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:59:37.721597 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:37.721511    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"
Jul 28 20:59:37.721748 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:37.721516    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5" event={"ID":"56791520-4e95-4361-84c5-c48da97a1f7a","Type":"ContainerDied","Data":"52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c"}
Jul 28 20:59:37.721748 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:37.721639    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52733825ec73132cfd50915d960943ea85afebfadecd4f82ba4aab35374c2f8c"
Jul 28 20:59:48.981172 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:48.981132    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:59:48.981733 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:48.981417    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" podUID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" containerName="maas-api" containerID="cri-o://d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a" gracePeriod=30
Jul 28 20:59:49.011213 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.011181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:49.111721 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.111678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:49.181376 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.181338    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"]
Jul 28 20:59:49.185480 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.185064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:49.186306 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.186282    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a8387f77-nwdp5"]
Jul 28 20:59:49.246667 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.246641    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:59:49.248996 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.248974    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:49.300664 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.300633    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56791520-4e95-4361-84c5-c48da97a1f7a" path="/var/lib/kubelet/pods/56791520-4e95-4361-84c5-c48da97a1f7a/volumes"
Jul 28 20:59:49.310754 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.310720    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca\") pod \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") "
Jul 28 20:59:49.310913 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.310820    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls\") pod \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") "
Jul 28 20:59:49.310977 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.310920    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tdqh\" (UniqueName: \"kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh\") pod \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\" (UID: \"b576a578-b4d7-4db2-ad5d-19b03b6f79c5\") "
Jul 28 20:59:49.311059 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.311037    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b576a578-b4d7-4db2-ad5d-19b03b6f79c5" (UID: "b576a578-b4d7-4db2-ad5d-19b03b6f79c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:59:49.311184 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.311168    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:59:49.312960 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.312933    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b576a578-b4d7-4db2-ad5d-19b03b6f79c5" (UID: "b576a578-b4d7-4db2-ad5d-19b03b6f79c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:59:49.313061 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.312987    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh" (OuterVolumeSpecName: "kube-api-access-9tdqh") pod "b576a578-b4d7-4db2-ad5d-19b03b6f79c5" (UID: "b576a578-b4d7-4db2-ad5d-19b03b6f79c5"). InnerVolumeSpecName "kube-api-access-9tdqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:59:49.412523 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.412486    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:59:49.412523 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.412516    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tdqh\" (UniqueName: \"kubernetes.io/projected/b576a578-b4d7-4db2-ad5d-19b03b6f79c5-kube-api-access-9tdqh\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 20:59:49.789758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.789721    2576 generic.go:356] "Generic (PLEG): container finished" podID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" containerID="d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a" exitCode=0
Jul 28 20:59:49.789758 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.789761    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"
Jul 28 20:59:49.789976 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.789807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" event={"ID":"b576a578-b4d7-4db2-ad5d-19b03b6f79c5","Type":"ContainerDied","Data":"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"}
Jul 28 20:59:49.789976 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.789858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g" event={"ID":"b576a578-b4d7-4db2-ad5d-19b03b6f79c5","Type":"ContainerDied","Data":"1fb34d63881f346912873eabd96f945539eedbe6fcca01ecae7a11bb86200209"}
Jul 28 20:59:49.789976 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.789884    2576 scope.go:117] "RemoveContainer" containerID="d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"
Jul 28 20:59:49.800276 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.800255    2576 scope.go:117] "RemoveContainer" containerID="d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"
Jul 28 20:59:49.800549 ip-10-0-139-131 kubenswrapper[2576]: E0728 20:59:49.800531    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a\": container with ID starting with d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a not found: ID does not exist" containerID="d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"
Jul 28 20:59:49.800631 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.800556    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a"} err="failed to get container status \"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a\": rpc error: code = NotFound desc = could not find container \"d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a\": container with ID starting with d91b08e9817d388f460c4e8eb10d5e870b4a8e0f4bef001e3616025b1eed2a2a not found: ID does not exist"
Jul 28 20:59:49.812256 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.812229    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:59:49.817152 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:49.817130    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a8387f77-5f6c4df6f8-fqt4g"]
Jul 28 20:59:51.300457 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:51.300421    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" path="/var/lib/kubelet/pods/b576a578-b4d7-4db2-ad5d-19b03b6f79c5/volumes"
Jul 28 20:59:52.636901 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:52.636864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 20:59:57.721175 ip-10-0-139-131 kubenswrapper[2576]: I0728 20:59:57.721134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:00.151408 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.151338    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"]
Jul 28 21:00:00.151943 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.151922    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" containerName="maas-api"
Jul 28 21:00:00.152020 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.151947    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" containerName="maas-api"
Jul 28 21:00:00.152020 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.151960    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56791520-4e95-4361-84c5-c48da97a1f7a" containerName="revoke-keys"
Jul 28 21:00:00.152020 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.151968    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56791520-4e95-4361-84c5-c48da97a1f7a" containerName="revoke-keys"
Jul 28 21:00:00.152174 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.152125    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56791520-4e95-4361-84c5-c48da97a1f7a" containerName="revoke-keys"
Jul 28 21:00:00.152174 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.152141    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b576a578-b4d7-4db2-ad5d-19b03b6f79c5" containerName="maas-api"
Jul 28 21:00:00.156613 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.156589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"
Jul 28 21:00:00.160514 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.160492    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:00:00.164695 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.164670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"]
Jul 28 21:00:00.167744 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.167720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"
Jul 28 21:00:00.313497 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.313469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"]
Jul 28 21:00:00.316188 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:00:00.316159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccbeb55_89af_4d55_9941_34c6ce50b8e5.slice/crio-6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79 WatchSource:0}: Error finding container 6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79: Status 404 returned error can't find the container with id 6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79
Jul 28 21:00:00.858499 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.858401    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928" event={"ID":"bccbeb55-89af-4d55-9941-34c6ce50b8e5","Type":"ContainerStarted","Data":"1fd4b417bbb885c407eb943bc14ae32545fdcb700cba344374e342ce3a9dd264"}
Jul 28 21:00:00.858499 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.858478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928" event={"ID":"bccbeb55-89af-4d55-9941-34c6ce50b8e5","Type":"ContainerStarted","Data":"6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79"}
Jul 28 21:00:00.876080 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:00.876022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928" podStartSLOduration=0.876006209 podStartE2EDuration="876.006209ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:00:00.873812712 +0000 UTC m=+2096.058917417" watchObservedRunningTime="2026-07-28 21:00:00.876006209 +0000 UTC m=+2096.061110915"
Jul 28 21:00:01.865194 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:01.865156    2576 generic.go:356] "Generic (PLEG): container finished" podID="bccbeb55-89af-4d55-9941-34c6ce50b8e5" containerID="1fd4b417bbb885c407eb943bc14ae32545fdcb700cba344374e342ce3a9dd264" exitCode=0
Jul 28 21:00:01.865622 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:01.865251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928" event={"ID":"bccbeb55-89af-4d55-9941-34c6ce50b8e5","Type":"ContainerDied","Data":"1fd4b417bbb885c407eb943bc14ae32545fdcb700cba344374e342ce3a9dd264"}
Jul 28 21:00:03.007554 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:03.007529    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"
Jul 28 21:00:03.878461 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:03.878426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928" event={"ID":"bccbeb55-89af-4d55-9941-34c6ce50b8e5","Type":"ContainerDied","Data":"6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79"}
Jul 28 21:00:03.878647 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:03.878478    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6afbb12e4cf38a6477946ec95730b4cb1fb542383b85fae8400cd45b60790e79"
Jul 28 21:00:03.878647 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:03.878432    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754540-zr928"
Jul 28 21:00:05.528327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:05.528293    2576 scope.go:117] "RemoveContainer" containerID="bb46e30f3fd37b643563451e6f94ac8a451e4050d96213496d8a45d5c465a647"
Jul 28 21:00:05.537166 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:05.537141    2576 scope.go:117] "RemoveContainer" containerID="30f4d951acedac717a969a802363b2393da326b30ee48cfd849c859d182cc455"
Jul 28 21:00:34.077475 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:34.077441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:34.979109 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:34.979071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:40.842985 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:40.842951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:40.911003 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:40.910965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:41.119184 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:41.119089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:41.376996 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:41.376908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:42.496758 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:42.496726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:42.653075 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:42.653036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:42.862720 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:42.862635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:43.503271 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:43.503237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:43.603743 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:43.603700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:43.706053 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:43.706018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:45.124943 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:45.124906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:50.847862 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:50.847827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:50.949359 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:50.949322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:51.552052 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:51.551982    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:57.924314 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:57.924232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:58.036116 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:58.036081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:00:58.132834 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:00:58.132800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:05.570957 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:05.570922    2576 scope.go:117] "RemoveContainer" containerID="02ea024caa7e490f882b34731ef9d7d21c64a731fd85ec8a505af3d695ee597d"
Jul 28 21:01:08.707229 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:08.707184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:09.079208 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:09.079163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:30.057013 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:30.056975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:36.843392 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:36.843329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:36.907724 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:36.907694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:37.217114 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:37.217024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:37.732596 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:37.732559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:37.876159 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:37.876125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:40.846867 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:40.846833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:40.945276 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:40.945242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:41.038537 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.038499    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:01:41.039133 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.039110    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bccbeb55-89af-4d55-9941-34c6ce50b8e5" containerName="cleanup"
Jul 28 21:01:41.039255 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.039135    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bccbeb55-89af-4d55-9941-34c6ce50b8e5" containerName="cleanup"
Jul 28 21:01:41.039324 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.039266    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bccbeb55-89af-4d55-9941-34c6ce50b8e5" containerName="cleanup"
Jul 28 21:01:41.042754 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.042731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.045246 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.045217    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7465ef-serving-cert\""
Jul 28 21:01:41.052858 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.051722    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:01:41.074378 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.074326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:41.133586 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.133438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.133784 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.133746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.133855 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.133812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2799j\" (UniqueName: \"kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.234918 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.234877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.235096 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.234948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.235096 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.234987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2799j\" (UniqueName: \"kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.235096 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:01:41.235047    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-serving-cert: secret "maas-api-e2e-inf-a-7465ef-serving-cert" not found
Jul 28 21:01:41.235234 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:01:41.235118    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls podName:ce08ce80-6ba0-4722-a769-226dc0b2a280 nodeName:}" failed. No retries permitted until 2026-07-28 21:01:41.735100508 +0000 UTC m=+2196.920205193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls") pod "maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" (UID: "ce08ce80-6ba0-4722-a769-226dc0b2a280") : secret "maas-api-e2e-inf-a-7465ef-serving-cert" not found
Jul 28 21:01:41.235704 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.235685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.243997 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.243953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2799j\" (UniqueName: \"kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.404029 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.403950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:41.665416 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.665308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:41.741085 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.741043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.743672 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.743647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") pod \"maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:41.763567 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.763538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:01:41.957148 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:41.957060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:42.096879 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:42.096847    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:01:42.098431 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:01:42.098397    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce08ce80_6ba0_4722_a769_226dc0b2a280.slice/crio-5fc71dcfedf450e63b495f3b22a0fbe7fae14a697e129a43acd0d2cf4edc0c35 WatchSource:0}: Error finding container 5fc71dcfedf450e63b495f3b22a0fbe7fae14a697e129a43acd0d2cf4edc0c35: Status 404 returned error can't find the container with id 5fc71dcfedf450e63b495f3b22a0fbe7fae14a697e129a43acd0d2cf4edc0c35
Jul 28 21:01:42.437914 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:42.437879    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" event={"ID":"ce08ce80-6ba0-4722-a769-226dc0b2a280","Type":"ContainerStarted","Data":"5fc71dcfedf450e63b495f3b22a0fbe7fae14a697e129a43acd0d2cf4edc0c35"}
Jul 28 21:01:43.446848 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:43.446811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" event={"ID":"ce08ce80-6ba0-4722-a769-226dc0b2a280","Type":"ContainerStarted","Data":"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"}
Jul 28 21:01:43.447256 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:43.446879    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:43.464085 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:43.464036    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" podStartSLOduration=2.160010131 podStartE2EDuration="2.464021048s" totalImagesPullingTime="304.010917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:01:43.461886926 +0000 UTC m=+2198.646991642" watchObservedRunningTime="2026-07-28 21:01:43.464021048 +0000 UTC m=+2198.649125754"
Jul 28 21:01:49.458772 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:49.458735    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:01:58.435245 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:01:58.435206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:05.833127 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:05.833086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:06.195013 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:06.194931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:06.578128 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:06.578094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:06.863334 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:06.863253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:10.089489 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.089447    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:02:10.093086 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.093067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.095580 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.095558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3d8bda-serving-cert\""
Jul 28 21:02:10.101450 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.101425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtm2p\" (UniqueName: \"kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.101581 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.101470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.101647 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.101615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.103784 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.103735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:02:10.202311 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.202278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.202522 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.202327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtm2p\" (UniqueName: \"kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.202522 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.202352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.202522 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:02:10.202466    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-serving-cert: secret "maas-api-e2e-inf-b-3d8bda-serving-cert" not found
Jul 28 21:02:10.202700 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:02:10.202552    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls podName:b25ec02b-6df8-45ed-a9ea-e1272be7af7b nodeName:}" failed. No retries permitted until 2026-07-28 21:02:10.702529082 +0000 UTC m=+2225.887633792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls") pod "maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" (UID: "b25ec02b-6df8-45ed-a9ea-e1272be7af7b") : secret "maas-api-e2e-inf-b-3d8bda-serving-cert" not found
Jul 28 21:02:10.202910 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.202893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.212872 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.212851    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtm2p\" (UniqueName: \"kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.707553 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.707519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:10.709866 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:10.709845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:11.005036 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:11.004998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:11.155280 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:11.155245    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:02:11.157224 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:02:11.157177    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25ec02b_6df8_45ed_a9ea_e1272be7af7b.slice/crio-924025a8384b82f1fbd093be7e17418b453a79117996b4f77f5bd83edbb3ecbf WatchSource:0}: Error finding container 924025a8384b82f1fbd093be7e17418b453a79117996b4f77f5bd83edbb3ecbf: Status 404 returned error can't find the container with id 924025a8384b82f1fbd093be7e17418b453a79117996b4f77f5bd83edbb3ecbf
Jul 28 21:02:11.484557 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:11.484472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:11.545429 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:11.545385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:11.603801 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:11.603770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" event={"ID":"b25ec02b-6df8-45ed-a9ea-e1272be7af7b","Type":"ContainerStarted","Data":"924025a8384b82f1fbd093be7e17418b453a79117996b4f77f5bd83edbb3ecbf"}
Jul 28 21:02:12.611626 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:12.611584    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" event={"ID":"b25ec02b-6df8-45ed-a9ea-e1272be7af7b","Type":"ContainerStarted","Data":"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"}
Jul 28 21:02:12.612051 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:12.611706    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:12.630078 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:12.630022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" podStartSLOduration=2.2439422159999998 podStartE2EDuration="2.63000013s" totalImagesPullingTime="386.057914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:02:12.627927069 +0000 UTC m=+2227.813031775" watchObservedRunningTime="2026-07-28 21:02:12.63000013 +0000 UTC m=+2227.815104838"
Jul 28 21:02:18.623350 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:18.623313    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:02:35.029834 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:35.029733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:37.433427 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:37.433393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:02:55.432858 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:02:55.432821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:39.534409 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:39.534333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:40.522676 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.522634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:40.644151 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.644107    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"]
Jul 28 21:03:40.647735 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.647712    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:40.649980 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.649959    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:03:40.656564 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.655621    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"]
Jul 28 21:03:40.765028 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.764982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6\" (UID: \"b7540484-ea34-4ee1-842c-3f37c779945d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:40.866439 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.866336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6\" (UID: \"b7540484-ea34-4ee1-842c-3f37c779945d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:40.866991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.866968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6\" (UID: \"b7540484-ea34-4ee1-842c-3f37c779945d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:40.961064 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:40.961028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:41.107269 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:41.107237    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"]
Jul 28 21:03:41.108556 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:03:41.108531    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7540484_ea34_4ee1_842c_3f37c779945d.slice/crio-85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2 WatchSource:0}: Error finding container 85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2: Status 404 returned error can't find the container with id 85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2
Jul 28 21:03:41.130139 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:41.130101    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6" event={"ID":"b7540484-ea34-4ee1-842c-3f37c779945d","Type":"ContainerStarted","Data":"85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2"}
Jul 28 21:03:42.137593 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:42.137491    2576 generic.go:356] "Generic (PLEG): container finished" podID="b7540484-ea34-4ee1-842c-3f37c779945d" containerID="b6799246825cfbe42842a1925da3ae67138236618e8e6cd73f6a23218f24ba49" exitCode=0
Jul 28 21:03:42.137593 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:42.137577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6" event={"ID":"b7540484-ea34-4ee1-842c-3f37c779945d","Type":"ContainerDied","Data":"b6799246825cfbe42842a1925da3ae67138236618e8e6cd73f6a23218f24ba49"}
Jul 28 21:03:43.280338 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:43.280314    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:43.387540 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:43.387502    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca\") pod \"b7540484-ea34-4ee1-842c-3f37c779945d\" (UID: \"b7540484-ea34-4ee1-842c-3f37c779945d\") "
Jul 28 21:03:43.387973 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:43.387934    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7540484-ea34-4ee1-842c-3f37c779945d" (UID: "b7540484-ea34-4ee1-842c-3f37c779945d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:03:43.489228 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:43.489153    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7540484-ea34-4ee1-842c-3f37c779945d-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:03:44.152413 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:44.152357    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"
Jul 28 21:03:44.152413 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:44.152391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6" event={"ID":"b7540484-ea34-4ee1-842c-3f37c779945d","Type":"ContainerDied","Data":"85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2"}
Jul 28 21:03:44.152637 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:44.152450    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85c3c0dfd0143637ca01780b16fb691d3de157f181d625749e51161a389600e2"
Jul 28 21:03:51.091232 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.091193    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:03:51.091680 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.091501    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" podUID="ce08ce80-6ba0-4722-a769-226dc0b2a280" containerName="maas-api" containerID="cri-o://2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95" gracePeriod=30
Jul 28 21:03:51.290785 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.290707    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"]
Jul 28 21:03:51.293315 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.293294    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7465ef-r66b6"]
Jul 28 21:03:51.301171 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.301145    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7540484-ea34-4ee1-842c-3f37c779945d" path="/var/lib/kubelet/pods/b7540484-ea34-4ee1-842c-3f37c779945d/volumes"
Jul 28 21:03:51.542187 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.542159    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:03:51.610575 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.610521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:51.658321 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.658192    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") pod \"ce08ce80-6ba0-4722-a769-226dc0b2a280\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") "
Jul 28 21:03:51.658639 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.658392    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca\") pod \"ce08ce80-6ba0-4722-a769-226dc0b2a280\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") "
Jul 28 21:03:51.658639 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.658432    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2799j\" (UniqueName: \"kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j\") pod \"ce08ce80-6ba0-4722-a769-226dc0b2a280\" (UID: \"ce08ce80-6ba0-4722-a769-226dc0b2a280\") "
Jul 28 21:03:51.659084 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.659058    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce08ce80-6ba0-4722-a769-226dc0b2a280" (UID: "ce08ce80-6ba0-4722-a769-226dc0b2a280"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:03:51.660509 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.660481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce08ce80-6ba0-4722-a769-226dc0b2a280" (UID: "ce08ce80-6ba0-4722-a769-226dc0b2a280"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:03:51.660801 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.660786    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j" (OuterVolumeSpecName: "kube-api-access-2799j") pod "ce08ce80-6ba0-4722-a769-226dc0b2a280" (UID: "ce08ce80-6ba0-4722-a769-226dc0b2a280"). InnerVolumeSpecName "kube-api-access-2799j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:03:51.672012 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.671988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:51.759775 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.759744    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce08ce80-6ba0-4722-a769-226dc0b2a280-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:03:51.759775 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.759771    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2799j\" (UniqueName: \"kubernetes.io/projected/ce08ce80-6ba0-4722-a769-226dc0b2a280-kube-api-access-2799j\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:03:51.759775 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:51.759782    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce08ce80-6ba0-4722-a769-226dc0b2a280-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:03:52.176884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.176850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:52.198921 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.198884    2576 generic.go:356] "Generic (PLEG): container finished" podID="ce08ce80-6ba0-4722-a769-226dc0b2a280" containerID="2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95" exitCode=0
Jul 28 21:03:52.199056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.198930    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"
Jul 28 21:03:52.199056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.198970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" event={"ID":"ce08ce80-6ba0-4722-a769-226dc0b2a280","Type":"ContainerDied","Data":"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"}
Jul 28 21:03:52.199056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.199022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j" event={"ID":"ce08ce80-6ba0-4722-a769-226dc0b2a280","Type":"ContainerDied","Data":"5fc71dcfedf450e63b495f3b22a0fbe7fae14a697e129a43acd0d2cf4edc0c35"}
Jul 28 21:03:52.199056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.199044    2576 scope.go:117] "RemoveContainer" containerID="2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"
Jul 28 21:03:52.208633 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.208615    2576 scope.go:117] "RemoveContainer" containerID="2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"
Jul 28 21:03:52.208892 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:03:52.208872    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95\": container with ID starting with 2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95 not found: ID does not exist" containerID="2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"
Jul 28 21:03:52.208961 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.208899    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95"} err="failed to get container status \"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95\": rpc error: code = NotFound desc = could not find container \"2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95\": container with ID starting with 2820a8763f32663e3664bd178024ec0ab8ada14492ed78a6cef6a2f13bccbf95 not found: ID does not exist"
Jul 28 21:03:52.220550 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.220525    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:03:52.224027 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:52.224003    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7465ef-68df6649d8-klx9j"]
Jul 28 21:03:53.300591 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:53.300560    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce08ce80-6ba0-4722-a769-226dc0b2a280" path="/var/lib/kubelet/pods/ce08ce80-6ba0-4722-a769-226dc0b2a280/volumes"
Jul 28 21:03:57.735303 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:57.735224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:03:58.372014 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.371982    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"]
Jul 28 21:03:58.372374 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372353    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7540484-ea34-4ee1-842c-3f37c779945d" containerName="revoke-keys"
Jul 28 21:03:58.372425 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372380    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7540484-ea34-4ee1-842c-3f37c779945d" containerName="revoke-keys"
Jul 28 21:03:58.372425 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372400    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce08ce80-6ba0-4722-a769-226dc0b2a280" containerName="maas-api"
Jul 28 21:03:58.372425 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372405    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce08ce80-6ba0-4722-a769-226dc0b2a280" containerName="maas-api"
Jul 28 21:03:58.372523 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372472    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce08ce80-6ba0-4722-a769-226dc0b2a280" containerName="maas-api"
Jul 28 21:03:58.372523 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.372479    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7540484-ea34-4ee1-842c-3f37c779945d" containerName="revoke-keys"
Jul 28 21:03:58.376945 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.376926    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:03:58.379311 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.379286    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:03:58.384116 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.384085    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"]
Jul 28 21:03:58.520893 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.520862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg\" (UID: \"7eaadd12-70a3-471e-ae7c-769890488d27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:03:58.622282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.622203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg\" (UID: \"7eaadd12-70a3-471e-ae7c-769890488d27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:03:58.622814 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.622795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg\" (UID: \"7eaadd12-70a3-471e-ae7c-769890488d27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:03:58.690291 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.690257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:03:58.831117 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:58.831084    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"]
Jul 28 21:03:58.832728 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:03:58.832702    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eaadd12_70a3_471e_ae7c_769890488d27.slice/crio-2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3 WatchSource:0}: Error finding container 2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3: Status 404 returned error can't find the container with id 2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3
Jul 28 21:03:59.244056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:59.244018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg" event={"ID":"7eaadd12-70a3-471e-ae7c-769890488d27","Type":"ContainerStarted","Data":"49b8ee9c3a59403569ef78900c6b7733464f3cb8877aa4077ee50a366afcc319"}
Jul 28 21:03:59.244244 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:59.244082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg" event={"ID":"7eaadd12-70a3-471e-ae7c-769890488d27","Type":"ContainerStarted","Data":"2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3"}
Jul 28 21:03:59.263008 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:03:59.262950    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg" podStartSLOduration=1.2629341570000001 podStartE2EDuration="1.262934157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:03:59.260769715 +0000 UTC m=+2334.445874422" watchObservedRunningTime="2026-07-28 21:03:59.262934157 +0000 UTC m=+2334.448038907"
Jul 28 21:04:00.251441 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:00.251405    2576 generic.go:356] "Generic (PLEG): container finished" podID="7eaadd12-70a3-471e-ae7c-769890488d27" containerID="49b8ee9c3a59403569ef78900c6b7733464f3cb8877aa4077ee50a366afcc319" exitCode=0
Jul 28 21:04:00.251833 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:00.251486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg" event={"ID":"7eaadd12-70a3-471e-ae7c-769890488d27","Type":"ContainerDied","Data":"49b8ee9c3a59403569ef78900c6b7733464f3cb8877aa4077ee50a366afcc319"}
Jul 28 21:04:00.336260 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:00.336223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:01.392113 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:01.392090    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:04:01.550927 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:01.550829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca\") pod \"7eaadd12-70a3-471e-ae7c-769890488d27\" (UID: \"7eaadd12-70a3-471e-ae7c-769890488d27\") "
Jul 28 21:04:01.551189 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:01.551162    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7eaadd12-70a3-471e-ae7c-769890488d27" (UID: "7eaadd12-70a3-471e-ae7c-769890488d27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:04:01.652004 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:01.651967    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaadd12-70a3-471e-ae7c-769890488d27-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:04:02.263483 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:02.263452    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"
Jul 28 21:04:02.263667 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:02.263451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg" event={"ID":"7eaadd12-70a3-471e-ae7c-769890488d27","Type":"ContainerDied","Data":"2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3"}
Jul 28 21:04:02.263667 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:02.263588    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d31531e1155bde08080d3a497e9152fee561e5573b9487bb239915c64dfd7c3"
Jul 28 21:04:05.612038 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:05.612004    2576 scope.go:117] "RemoveContainer" containerID="c8af830eade088a81d991f695d8fa7cdca301145612c5c9ef735799fe8db0e7c"
Jul 28 21:04:06.709643 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:06.709610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:08.884040 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:08.884001    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:04:08.884508 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:08.884210    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" podUID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" containerName="maas-api" containerID="cri-o://04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82" gracePeriod=30
Jul 28 21:04:08.911565 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:08.911537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:08.987396 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:08.987351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:09.061228 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.061159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:09.100290 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.100253    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"]
Jul 28 21:04:09.102695 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.102670    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3d8bda-prffg"]
Jul 28 21:04:09.126654 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.126613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:09.141947 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.141884    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:04:09.300769 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.300733    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eaadd12-70a3-471e-ae7c-769890488d27" path="/var/lib/kubelet/pods/7eaadd12-70a3-471e-ae7c-769890488d27/volumes"
Jul 28 21:04:09.303732 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.303708    2576 generic.go:356] "Generic (PLEG): container finished" podID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" containerID="04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82" exitCode=0
Jul 28 21:04:09.303882 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.303735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" event={"ID":"b25ec02b-6df8-45ed-a9ea-e1272be7af7b","Type":"ContainerDied","Data":"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"}
Jul 28 21:04:09.303882 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.303759    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"
Jul 28 21:04:09.303882 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.303784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28" event={"ID":"b25ec02b-6df8-45ed-a9ea-e1272be7af7b","Type":"ContainerDied","Data":"924025a8384b82f1fbd093be7e17418b453a79117996b4f77f5bd83edbb3ecbf"}
Jul 28 21:04:09.303882 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.303806    2576 scope.go:117] "RemoveContainer" containerID="04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"
Jul 28 21:04:09.312674 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.312657    2576 scope.go:117] "RemoveContainer" containerID="04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"
Jul 28 21:04:09.312916 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:04:09.312900    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82\": container with ID starting with 04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82 not found: ID does not exist" containerID="04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"
Jul 28 21:04:09.312981 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.312924    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82"} err="failed to get container status \"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82\": rpc error: code = NotFound desc = could not find container \"04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82\": container with ID starting with 04be56357103a40aed5531e41b3c737d1b8381a21a4e119cd156644ad6400c82 not found: ID does not exist"
Jul 28 21:04:09.324227 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.324204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtm2p\" (UniqueName: \"kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p\") pod \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") "
Jul 28 21:04:09.324310 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.324245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") pod \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") "
Jul 28 21:04:09.324349 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.324308    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca\") pod \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\" (UID: \"b25ec02b-6df8-45ed-a9ea-e1272be7af7b\") "
Jul 28 21:04:09.324699 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.324679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b25ec02b-6df8-45ed-a9ea-e1272be7af7b" (UID: "b25ec02b-6df8-45ed-a9ea-e1272be7af7b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:04:09.326232 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.326206    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p" (OuterVolumeSpecName: "kube-api-access-vtm2p") pod "b25ec02b-6df8-45ed-a9ea-e1272be7af7b" (UID: "b25ec02b-6df8-45ed-a9ea-e1272be7af7b"). InnerVolumeSpecName "kube-api-access-vtm2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:04:09.326321 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.326259    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b25ec02b-6df8-45ed-a9ea-e1272be7af7b" (UID: "b25ec02b-6df8-45ed-a9ea-e1272be7af7b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:04:09.425733 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.425646    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:04:09.425733 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.425677    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtm2p\" (UniqueName: \"kubernetes.io/projected/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-kube-api-access-vtm2p\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:04:09.425733 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.425687    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b25ec02b-6df8-45ed-a9ea-e1272be7af7b-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:04:09.586856 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.586823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:09.626770 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.626737    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:04:09.630699 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:09.630676    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d8bda-84c85bd66d-vlp28"]
Jul 28 21:04:11.300353 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:11.300317    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" path="/var/lib/kubelet/pods/b25ec02b-6df8-45ed-a9ea-e1272be7af7b/volumes"
Jul 28 21:04:15.243387 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:15.243336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:15.313838 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:15.313801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:15.528697 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:15.528614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:15.765443 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:15.765410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:16.334458 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:16.334422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:16.459997 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:16.459957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:16.753845 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:16.753806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:19.467111 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.467075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:19.549126 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549091    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:04:19.549677 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549653    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" containerName="maas-api"
Jul 28 21:04:19.549677 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549674    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" containerName="maas-api"
Jul 28 21:04:19.549884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549688    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eaadd12-70a3-471e-ae7c-769890488d27" containerName="revoke-keys"
Jul 28 21:04:19.549884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549698    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eaadd12-70a3-471e-ae7c-769890488d27" containerName="revoke-keys"
Jul 28 21:04:19.549884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549765    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eaadd12-70a3-471e-ae7c-769890488d27" containerName="revoke-keys"
Jul 28 21:04:19.549884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.549774    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25ec02b-6df8-45ed-a9ea-e1272be7af7b" containerName="maas-api"
Jul 28 21:04:19.550050 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.550014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:19.553087 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.553072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.555442 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.555417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8e6a5a-serving-cert\""
Jul 28 21:04:19.567311 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.567287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:04:19.608814 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.608782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.608988 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.608825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.608988 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.608856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjh5\" (UniqueName: \"kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.625029 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.624995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:19.710358 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.710318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.710581 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.710378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.710581 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:04:19.710472    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-serving-cert: secret "maas-api-e2e-api-a-8e6a5a-serving-cert" not found
Jul 28 21:04:19.710581 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.710540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjh5\" (UniqueName: \"kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.710581 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:04:19.710546    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls podName:bd2fcad0-2aba-4979-9889-898ded1066ab nodeName:}" failed. No retries permitted until 2026-07-28 21:04:20.210529703 +0000 UTC m=+2355.395634387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls") pod "maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" (UID: "bd2fcad0-2aba-4979-9889-898ded1066ab") : secret "maas-api-e2e-api-a-8e6a5a-serving-cert" not found
Jul 28 21:04:19.711093 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.711071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.719253 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.719188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjh5\" (UniqueName: \"kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:19.788563 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:19.788528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:20.214476 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.214434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:20.216905 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.216877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") pod \"maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:20.284979 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.284944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:20.367587 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.367547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:20.464107 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.464066    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:20.644700 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.644637    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:04:20.647622 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:04:20.647596    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2fcad0_2aba_4979_9889_898ded1066ab.slice/crio-9d9514604ca58b57e1b9451a0f5813f31bcda2908e787e23529b5059d5458307 WatchSource:0}: Error finding container 9d9514604ca58b57e1b9451a0f5813f31bcda2908e787e23529b5059d5458307: Status 404 returned error can't find the container with id 9d9514604ca58b57e1b9451a0f5813f31bcda2908e787e23529b5059d5458307
Jul 28 21:04:20.649115 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:20.649095    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:04:21.373149 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:21.373116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" event={"ID":"bd2fcad0-2aba-4979-9889-898ded1066ab","Type":"ContainerStarted","Data":"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"}
Jul 28 21:04:21.373340 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:21.373167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" event={"ID":"bd2fcad0-2aba-4979-9889-898ded1066ab","Type":"ContainerStarted","Data":"9d9514604ca58b57e1b9451a0f5813f31bcda2908e787e23529b5059d5458307"}
Jul 28 21:04:21.373340 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:21.373213    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:21.390820 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:21.390775    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" podStartSLOduration=2.07054255 podStartE2EDuration="2.390759069s" totalImagesPullingTime="320.216519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:04:21.38893308 +0000 UTC m=+2356.574037788" watchObservedRunningTime="2026-07-28 21:04:21.390759069 +0000 UTC m=+2356.575863774"
Jul 28 21:04:27.384326 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:27.384296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:04:44.509047 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:44.509014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:45.013335 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:45.013299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:45.427266 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:45.427152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:45.740908 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:45.740873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:48.633870 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.633829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:48.723672 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.723637    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:04:48.725515 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.725485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:48.727849 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.727827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.730032 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.730012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-fec7c6-serving-cert\""
Jul 28 21:04:48.740617 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.740593    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:04:48.797733 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.797691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:48.874526 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.874493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdlhp\" (UniqueName: \"kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.874713 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.874539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.874713 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.874624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.898762 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.898654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:48.975765 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.975719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdlhp\" (UniqueName: \"kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.975765 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.975774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.976022 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.975820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.976600 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.976573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.978597 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.978567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:48.984269 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:48.984244    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdlhp\" (UniqueName: \"kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp\") pod \"maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:49.040093 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:49.040051    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:49.193346 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:49.193320    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:04:49.194704 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:04:49.194678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902f8d9c_8272_4360_97b6_0b828c37d58e.slice/crio-31c58756ab447f39ac971e0956544adce65699e51d744a54982086de741820bb WatchSource:0}: Error finding container 31c58756ab447f39ac971e0956544adce65699e51d744a54982086de741820bb: Status 404 returned error can't find the container with id 31c58756ab447f39ac971e0956544adce65699e51d744a54982086de741820bb
Jul 28 21:04:49.427201 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:49.427156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:49.529161 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:49.529118    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:04:49.533534 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:49.533509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" event={"ID":"902f8d9c-8272-4360-97b6-0b828c37d58e","Type":"ContainerStarted","Data":"31c58756ab447f39ac971e0956544adce65699e51d744a54982086de741820bb"}
Jul 28 21:04:50.541257 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:50.541223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" event={"ID":"902f8d9c-8272-4360-97b6-0b828c37d58e","Type":"ContainerStarted","Data":"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"}
Jul 28 21:04:50.541820 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:50.541293    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:04:50.557680 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:50.557627    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" podStartSLOduration=2.161270734 podStartE2EDuration="2.557608487s" totalImagesPullingTime="396.337753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:04:50.556621356 +0000 UTC m=+2385.741726063" watchObservedRunningTime="2026-07-28 21:04:50.557608487 +0000 UTC m=+2385.742713184"
Jul 28 21:04:56.553273 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:04:56.553244    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:05:13.678091 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:13.678047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:16.776890 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.776848    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"]
Jul 28 21:05:16.780447 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.780428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:16.783031 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.783005    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:05:16.788272 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.788250    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"]
Jul 28 21:05:16.820574 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.820544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx\" (UID: \"7332572b-4bd5-4f48-adcd-b6417a6c0c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:16.921283 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.921244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx\" (UID: \"7332572b-4bd5-4f48-adcd-b6417a6c0c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:16.921843 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:16.921825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx\" (UID: \"7332572b-4bd5-4f48-adcd-b6417a6c0c5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:17.091959 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:17.091874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:17.227353 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:17.227324    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"]
Jul 28 21:05:17.228731 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:05:17.228704    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7332572b_4bd5_4f48_adcd_b6417a6c0c5b.slice/crio-27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0 WatchSource:0}: Error finding container 27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0: Status 404 returned error can't find the container with id 27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0
Jul 28 21:05:17.701566 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:17.701523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx" event={"ID":"7332572b-4bd5-4f48-adcd-b6417a6c0c5b","Type":"ContainerStarted","Data":"9b8ee6525359ad3e8773796ac614770ff55b465b5784beb811a47b8b49960b80"}
Jul 28 21:05:17.701750 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:17.701586    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx" event={"ID":"7332572b-4bd5-4f48-adcd-b6417a6c0c5b","Type":"ContainerStarted","Data":"27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0"}
Jul 28 21:05:17.720641 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:17.720544    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx" podStartSLOduration=1.720530726 podStartE2EDuration="1.720530726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:05:17.718912733 +0000 UTC m=+2412.904017441" watchObservedRunningTime="2026-07-28 21:05:17.720530726 +0000 UTC m=+2412.905635432"
Jul 28 21:05:18.708541 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:18.708504    2576 generic.go:356] "Generic (PLEG): container finished" podID="7332572b-4bd5-4f48-adcd-b6417a6c0c5b" containerID="9b8ee6525359ad3e8773796ac614770ff55b465b5784beb811a47b8b49960b80" exitCode=0
Jul 28 21:05:18.709012 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:18.708588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx" event={"ID":"7332572b-4bd5-4f48-adcd-b6417a6c0c5b","Type":"ContainerDied","Data":"9b8ee6525359ad3e8773796ac614770ff55b465b5784beb811a47b8b49960b80"}
Jul 28 21:05:19.851985 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:19.851957    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:20.047136 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.047103    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca\") pod \"7332572b-4bd5-4f48-adcd-b6417a6c0c5b\" (UID: \"7332572b-4bd5-4f48-adcd-b6417a6c0c5b\") "
Jul 28 21:05:20.047514 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.047488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7332572b-4bd5-4f48-adcd-b6417a6c0c5b" (UID: "7332572b-4bd5-4f48-adcd-b6417a6c0c5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:20.148147 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.148111    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7332572b-4bd5-4f48-adcd-b6417a6c0c5b-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:20.721398 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.721352    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"
Jul 28 21:05:20.721398 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.721391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx" event={"ID":"7332572b-4bd5-4f48-adcd-b6417a6c0c5b","Type":"ContainerDied","Data":"27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0"}
Jul 28 21:05:20.721620 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:20.721441    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27458dcae99b5261d673546a9bad7d3512e6e1b842f131d2dca4a8e24d1136e0"
Jul 28 21:05:21.806807 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:21.806773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:21.856718 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:21.856682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.192268 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.192168    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:05:32.192760 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.192424    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerName="maas-api" containerID="cri-o://f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653" gracePeriod=30
Jul 28 21:05:32.214557 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.214531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.284866 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.284834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.351294 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.351242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.398411 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.398351    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"]
Jul 28 21:05:32.400762 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.400739    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-8e6a5a-j5ksx"]
Jul 28 21:05:32.407531 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.407502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.446558 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.446501    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:05:32.559308 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.559278    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbjh5\" (UniqueName: \"kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5\") pod \"bd2fcad0-2aba-4979-9889-898ded1066ab\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") "
Jul 28 21:05:32.559481 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.559315    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") pod \"bd2fcad0-2aba-4979-9889-898ded1066ab\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") "
Jul 28 21:05:32.559481 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.559355    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca\") pod \"bd2fcad0-2aba-4979-9889-898ded1066ab\" (UID: \"bd2fcad0-2aba-4979-9889-898ded1066ab\") "
Jul 28 21:05:32.559852 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.559818    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd2fcad0-2aba-4979-9889-898ded1066ab" (UID: "bd2fcad0-2aba-4979-9889-898ded1066ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:32.561542 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.561503    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd2fcad0-2aba-4979-9889-898ded1066ab" (UID: "bd2fcad0-2aba-4979-9889-898ded1066ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:05:32.561652 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.561542    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5" (OuterVolumeSpecName: "kube-api-access-zbjh5") pod "bd2fcad0-2aba-4979-9889-898ded1066ab" (UID: "bd2fcad0-2aba-4979-9889-898ded1066ab"). InnerVolumeSpecName "kube-api-access-zbjh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:05:32.661137 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.661108    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd2fcad0-2aba-4979-9889-898ded1066ab-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:32.661137 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.661133    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd2fcad0-2aba-4979-9889-898ded1066ab-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:32.661137 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.661143    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbjh5\" (UniqueName: \"kubernetes.io/projected/bd2fcad0-2aba-4979-9889-898ded1066ab-kube-api-access-zbjh5\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:32.788896 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.788857    2576 generic.go:356] "Generic (PLEG): container finished" podID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerID="f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653" exitCode=0
Jul 28 21:05:32.789100 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.788908    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"
Jul 28 21:05:32.789100 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.788943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" event={"ID":"bd2fcad0-2aba-4979-9889-898ded1066ab","Type":"ContainerDied","Data":"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"}
Jul 28 21:05:32.789100 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.789008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" event={"ID":"bd2fcad0-2aba-4979-9889-898ded1066ab","Type":"ContainerDied","Data":"9d9514604ca58b57e1b9451a0f5813f31bcda2908e787e23529b5059d5458307"}
Jul 28 21:05:32.789100 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.789038    2576 scope.go:117] "RemoveContainer" containerID="f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"
Jul 28 21:05:32.798649 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.798627    2576 scope.go:117] "RemoveContainer" containerID="f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"
Jul 28 21:05:32.798891 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:05:32.798873    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653\": container with ID starting with f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653 not found: ID does not exist" containerID="f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"
Jul 28 21:05:32.798958 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.798900    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653"} err="failed to get container status \"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653\": rpc error: code = NotFound desc = could not find container \"f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653\": container with ID starting with f6a132f3ba7f1e0dcd3dd781513dff0eb9fa316e446185501e85b58e1a231653 not found: ID does not exist"
Jul 28 21:05:32.811545 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.811521    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:05:32.812884 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.812859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:32.813970 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:32.813951    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh"]
Jul 28 21:05:33.300506 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:33.300476    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7332572b-4bd5-4f48-adcd-b6417a6c0c5b" path="/var/lib/kubelet/pods/7332572b-4bd5-4f48-adcd-b6417a6c0c5b/volumes"
Jul 28 21:05:33.300873 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:33.300857    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" path="/var/lib/kubelet/pods/bd2fcad0-2aba-4979-9889-898ded1066ab/volumes"
Jul 28 21:05:35.379817 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:35.379774    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8e6a5a-cddd6ccf6-xs9kh" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.83:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 21:05:38.735283 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:38.735248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:38.836379 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:38.836344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:39.005703 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.005676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:39.256357 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.256260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:39.408718 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.408682    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"]
Jul 28 21:05:39.409128 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409114    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerName="maas-api"
Jul 28 21:05:39.409186 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409130    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerName="maas-api"
Jul 28 21:05:39.409186 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409149    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7332572b-4bd5-4f48-adcd-b6417a6c0c5b" containerName="revoke-keys"
Jul 28 21:05:39.409186 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409154    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7332572b-4bd5-4f48-adcd-b6417a6c0c5b" containerName="revoke-keys"
Jul 28 21:05:39.409283 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409214    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7332572b-4bd5-4f48-adcd-b6417a6c0c5b" containerName="revoke-keys"
Jul 28 21:05:39.409283 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.409223    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd2fcad0-2aba-4979-9889-898ded1066ab" containerName="maas-api"
Jul 28 21:05:39.413729 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.413708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:39.416853 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.416833    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:05:39.435793 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.435768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"]
Jul 28 21:05:39.520672 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.520590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq\" (UID: \"d491adb8-a6f0-4bab-9856-f3898ae604cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:39.622140 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.622100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq\" (UID: \"d491adb8-a6f0-4bab-9856-f3898ae604cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:39.622758 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.622738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq\" (UID: \"d491adb8-a6f0-4bab-9856-f3898ae604cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:39.723854 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.723819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:39.868415 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:39.868386    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"]
Jul 28 21:05:39.869875 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:05:39.869847    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd491adb8_a6f0_4bab_9856_f3898ae604cd.slice/crio-6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7 WatchSource:0}: Error finding container 6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7: Status 404 returned error can't find the container with id 6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7
Jul 28 21:05:40.837632 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:40.837591    2576 generic.go:356] "Generic (PLEG): container finished" podID="d491adb8-a6f0-4bab-9856-f3898ae604cd" containerID="effa5efc30a85d896e9d88c1d54e01d824e13f11729128570f029931e4038c30" exitCode=0
Jul 28 21:05:40.837804 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:40.837639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq" event={"ID":"d491adb8-a6f0-4bab-9856-f3898ae604cd","Type":"ContainerDied","Data":"effa5efc30a85d896e9d88c1d54e01d824e13f11729128570f029931e4038c30"}
Jul 28 21:05:40.837804 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:40.837686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq" event={"ID":"d491adb8-a6f0-4bab-9856-f3898ae604cd","Type":"ContainerStarted","Data":"6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7"}
Jul 28 21:05:41.975271 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:41.975245    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:42.046236 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.046201    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca\") pod \"d491adb8-a6f0-4bab-9856-f3898ae604cd\" (UID: \"d491adb8-a6f0-4bab-9856-f3898ae604cd\") "
Jul 28 21:05:42.046707 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.046679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d491adb8-a6f0-4bab-9856-f3898ae604cd" (UID: "d491adb8-a6f0-4bab-9856-f3898ae604cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:42.147220 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.147135    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d491adb8-a6f0-4bab-9856-f3898ae604cd-maas-api-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:42.851608 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.851569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq" event={"ID":"d491adb8-a6f0-4bab-9856-f3898ae604cd","Type":"ContainerDied","Data":"6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7"}
Jul 28 21:05:42.851608 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.851592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"
Jul 28 21:05:42.851821 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:42.851630    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dbccf7f3e5b129f03df56a408eea31ada7269361983349cc24c2c5549ccd2c7"
Jul 28 21:05:54.842727 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:54.842686    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:05:54.843329 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:54.842948    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" podUID="902f8d9c-8272-4360-97b6-0b828c37d58e" containerName="maas-api" containerID="cri-o://00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd" gracePeriod=30
Jul 28 21:05:54.861976 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:54.861949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:54.963152 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:54.963122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:55.032824 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.032755    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"]
Jul 28 21:05:55.034657 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.034622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:55.035730 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.035706    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-fec7c6-dzjtq"]
Jul 28 21:05:55.099813 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.099735    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:05:55.099952 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.099874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:55.270200 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.270165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdlhp\" (UniqueName: \"kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp\") pod \"902f8d9c-8272-4360-97b6-0b828c37d58e\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") "
Jul 28 21:05:55.270397 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.270233    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca\") pod \"902f8d9c-8272-4360-97b6-0b828c37d58e\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") "
Jul 28 21:05:55.270397 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.270324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls\") pod \"902f8d9c-8272-4360-97b6-0b828c37d58e\" (UID: \"902f8d9c-8272-4360-97b6-0b828c37d58e\") "
Jul 28 21:05:55.270617 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.270595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "902f8d9c-8272-4360-97b6-0b828c37d58e" (UID: "902f8d9c-8272-4360-97b6-0b828c37d58e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:05:55.272277 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.272257    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "902f8d9c-8272-4360-97b6-0b828c37d58e" (UID: "902f8d9c-8272-4360-97b6-0b828c37d58e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:05:55.272394 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.272377    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp" (OuterVolumeSpecName: "kube-api-access-vdlhp") pod "902f8d9c-8272-4360-97b6-0b828c37d58e" (UID: "902f8d9c-8272-4360-97b6-0b828c37d58e"). InnerVolumeSpecName "kube-api-access-vdlhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:05:55.301999 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.301967    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d491adb8-a6f0-4bab-9856-f3898ae604cd" path="/var/lib/kubelet/pods/d491adb8-a6f0-4bab-9856-f3898ae604cd/volumes"
Jul 28 21:05:55.371084 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.370993    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdlhp\" (UniqueName: \"kubernetes.io/projected/902f8d9c-8272-4360-97b6-0b828c37d58e-kube-api-access-vdlhp\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:55.371084 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.371025    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/902f8d9c-8272-4360-97b6-0b828c37d58e-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:55.371084 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.371036    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/902f8d9c-8272-4360-97b6-0b828c37d58e-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:05:55.435165 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.435136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:05:55.932837 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.932800    2576 generic.go:356] "Generic (PLEG): container finished" podID="902f8d9c-8272-4360-97b6-0b828c37d58e" containerID="00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd" exitCode=0
Jul 28 21:05:55.933327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.932889    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"
Jul 28 21:05:55.933327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.932916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" event={"ID":"902f8d9c-8272-4360-97b6-0b828c37d58e","Type":"ContainerDied","Data":"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"}
Jul 28 21:05:55.933327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.932977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5" event={"ID":"902f8d9c-8272-4360-97b6-0b828c37d58e","Type":"ContainerDied","Data":"31c58756ab447f39ac971e0956544adce65699e51d744a54982086de741820bb"}
Jul 28 21:05:55.933327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.933006    2576 scope.go:117] "RemoveContainer" containerID="00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"
Jul 28 21:05:55.943905 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.943857    2576 scope.go:117] "RemoveContainer" containerID="00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"
Jul 28 21:05:55.944095 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:05:55.944072    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd\": container with ID starting with 00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd not found: ID does not exist" containerID="00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"
Jul 28 21:05:55.944168 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.944105    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd"} err="failed to get container status \"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd\": rpc error: code = NotFound desc = could not find container \"00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd\": container with ID starting with 00b2c9c9c367ca2f0c02361913ee77b94c90811383b4bdaa26d837c12d748bdd not found: ID does not exist"
Jul 28 21:05:55.953384 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.953337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:05:55.962502 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:55.962476    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-fec7c6-68f79fc897-gqhq5"]
Jul 28 21:05:57.300897 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:57.300862    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="902f8d9c-8272-4360-97b6-0b828c37d58e" path="/var/lib/kubelet/pods/902f8d9c-8272-4360-97b6-0b828c37d58e/volumes"
Jul 28 21:05:58.831988 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:05:58.831956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:01.165176 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:01.165139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:01.225319 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:01.225292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:01.492905 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:01.492876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:01.684112 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:01.684077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:02.109932 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:02.109898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:02.414572 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:02.414482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:02.726710 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:02.726621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.006479 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.006438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.106945 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.106910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.303115 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303020    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:06:05.303803 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303776    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902f8d9c-8272-4360-97b6-0b828c37d58e" containerName="maas-api"
Jul 28 21:06:05.303803 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303803    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="902f8d9c-8272-4360-97b6-0b828c37d58e" containerName="maas-api"
Jul 28 21:06:05.304078 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303823    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d491adb8-a6f0-4bab-9856-f3898ae604cd" containerName="revoke-keys"
Jul 28 21:06:05.304078 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d491adb8-a6f0-4bab-9856-f3898ae604cd" containerName="revoke-keys"
Jul 28 21:06:05.304078 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303925    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d491adb8-a6f0-4bab-9856-f3898ae604cd" containerName="revoke-keys"
Jul 28 21:06:05.304078 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.303939    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="902f8d9c-8272-4360-97b6-0b828c37d58e" containerName="maas-api"
Jul 28 21:06:05.308735 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.308709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:06:05.308863 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.308821    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.311239 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.311217    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-96b3c5-serving-cert\""
Jul 28 21:06:05.329564 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.329528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.395498 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.395465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.468831 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.468796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.469037 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.468876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.469037 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.468925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq8ks\" (UniqueName: \"kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.569936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.569837    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.569936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.569891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq8ks\" (UniqueName: \"kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.569936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.569934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.570618 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.570598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.572301 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.572275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.577672 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.577649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq8ks\" (UniqueName: \"kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks\") pod \"maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.619651 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.619613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:05.657813 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.657726    2576 scope.go:117] "RemoveContainer" containerID="5d5e573fff9796d6b73f3ddb5c94e0f10e2c17f4b6b3cc36a8929b004e29262a"
Jul 28 21:06:05.757860 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.757831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.769876 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.769846    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:06:05.771404 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:06:05.771345    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799608e8_cacf_4c35_aedf_1d7ff2f33853.slice/crio-95ee5477bdf91a1fa53a1819057f756f0d54ab52609e73cc13e9bcce3d68ff28 WatchSource:0}: Error finding container 95ee5477bdf91a1fa53a1819057f756f0d54ab52609e73cc13e9bcce3d68ff28: Status 404 returned error can't find the container with id 95ee5477bdf91a1fa53a1819057f756f0d54ab52609e73cc13e9bcce3d68ff28
Jul 28 21:06:05.835624 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.835508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:05.882600 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.882570    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:06:05.887663 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.887640    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:05.890464 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.890433    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-96b3c5\""
Jul 28 21:06:05.890663 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.890481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-96b3c5-dockercfg-tl8xg\""
Jul 28 21:06:05.893246 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.893220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:06:05.972414 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.972359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:05.972585 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.972495    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfh8w\" (UniqueName: \"kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:05.998406 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:05.998354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" event={"ID":"799608e8-cacf-4c35-aedf-1d7ff2f33853","Type":"ContainerStarted","Data":"95ee5477bdf91a1fa53a1819057f756f0d54ab52609e73cc13e9bcce3d68ff28"}
Jul 28 21:06:06.073331 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.073297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfh8w\" (UniqueName: \"kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:06.073809 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.073480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:06.074146 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.074126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:06.081085 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.081051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfh8w\" (UniqueName: \"kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w\") pod \"payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") " pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:06.200080 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.199972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:06.343126 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:06.342998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:06:06.345983 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:06:06.345955    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971f72e8_ec1d_4d75_9f45_5a2ae28394ba.slice/crio-4ecfd58e58b8bae27833b5982561791d42a70ba583ae9ba05f77f9e71b479428 WatchSource:0}: Error finding container 4ecfd58e58b8bae27833b5982561791d42a70ba583ae9ba05f77f9e71b479428: Status 404 returned error can't find the container with id 4ecfd58e58b8bae27833b5982561791d42a70ba583ae9ba05f77f9e71b479428
Jul 28 21:06:07.005827 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.005786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" event={"ID":"971f72e8-ec1d-4d75-9f45-5a2ae28394ba","Type":"ContainerStarted","Data":"003842b4c89d12db4d8787d2bfbe601e0e6817bd9c6cc8b51033cc204222ff08"}
Jul 28 21:06:07.006036 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.005846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" event={"ID":"971f72e8-ec1d-4d75-9f45-5a2ae28394ba","Type":"ContainerStarted","Data":"4ecfd58e58b8bae27833b5982561791d42a70ba583ae9ba05f77f9e71b479428"}
Jul 28 21:06:07.006036 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.005978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:07.007465 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.007433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" event={"ID":"799608e8-cacf-4c35-aedf-1d7ff2f33853","Type":"ContainerStarted","Data":"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"}
Jul 28 21:06:07.007671 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.007552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:07.021048 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.020999    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" podStartSLOduration=2.02098437 podStartE2EDuration="2.02098437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:06:07.019131183 +0000 UTC m=+2462.204235905" watchObservedRunningTime="2026-07-28 21:06:07.02098437 +0000 UTC m=+2462.206089076"
Jul 28 21:06:07.034467 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:07.034424    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" podStartSLOduration=1.443470274 podStartE2EDuration="2.034413044s" totalImagesPullingTime="590.94277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:06:07.032996134 +0000 UTC m=+2462.218100841" watchObservedRunningTime="2026-07-28 21:06:07.034413044 +0000 UTC m=+2462.219517749"
Jul 28 21:06:13.018463 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:13.018432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:06:18.015353 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:18.015318    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:06:30.044309 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:30.044271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:30.498931 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:30.498897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:30.799697 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:30.799608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:31.230911 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:31.230825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.115037 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.115001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.203723 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.203680    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:06:34.208109 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.208088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.210625 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.210595    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-4e7a7a-serving-cert\""
Jul 28 21:06:34.215639 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.215606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.216469 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.216431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:06:34.318776 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.318742    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.318965 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.318843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.318965 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.318873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz8ks\" (UniqueName: \"kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.395082 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.394985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.420250 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.420210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.420250 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.420256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz8ks\" (UniqueName: \"kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.420519 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.420300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.420966 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.420947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.422640 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.422620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.430439 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.430418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz8ks\" (UniqueName: \"kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks\") pod \"maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.521899 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.521857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:34.536657 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.536623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.705156 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.705127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:06:34.706546 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:06:34.706517    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba2ad54_b600_4fc8_8025_8a21be163cc1.slice/crio-bbd8be6bdc123a35777061cce2b84cc1c9073c55f44e32d55d7e80358544c785 WatchSource:0}: Error finding container bbd8be6bdc123a35777061cce2b84cc1c9073c55f44e32d55d7e80358544c785: Status 404 returned error can't find the container with id bbd8be6bdc123a35777061cce2b84cc1c9073c55f44e32d55d7e80358544c785
Jul 28 21:06:34.874775 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.874745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:34.973476 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:34.973393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:35.171351 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:35.171318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" event={"ID":"8ba2ad54-b600-4fc8-8025-8a21be163cc1","Type":"ContainerStarted","Data":"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"}
Jul 28 21:06:35.171857 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:35.171400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" event={"ID":"8ba2ad54-b600-4fc8-8025-8a21be163cc1","Type":"ContainerStarted","Data":"bbd8be6bdc123a35777061cce2b84cc1c9073c55f44e32d55d7e80358544c785"}
Jul 28 21:06:35.171857 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:35.171431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:35.187911 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:35.187852    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" podStartSLOduration=0.845779101 podStartE2EDuration="1.187833804s" totalImagesPullingTime="342.054703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:06:35.186868176 +0000 UTC m=+2490.371972883" watchObservedRunningTime="2026-07-28 21:06:35.187833804 +0000 UTC m=+2490.372938511"
Jul 28 21:06:41.182546 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:41.182511    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:06:45.952210 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:45.952161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:57.236676 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:57.236587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:06:59.137242 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:06:59.137210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:07:08.564099 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:07:08.564062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:07:33.824886 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:07:33.824837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:09:38.566751 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:09:38.566715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:10:05.729050 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:10:05.729016    2576 scope.go:117] "RemoveContainer" containerID="b6799246825cfbe42842a1925da3ae67138236618e8e6cd73f6a23218f24ba49"
Jul 28 21:10:05.737915 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:10:05.737892    2576 scope.go:117] "RemoveContainer" containerID="49b8ee9c3a59403569ef78900c6b7733464f3cb8877aa4077ee50a366afcc319"
Jul 28 21:12:04.115282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:12:04.115200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:12:05.005242 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:12:05.005214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:12:05.771331 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:12:05.771298    2576 scope.go:117] "RemoveContainer" containerID="effa5efc30a85d896e9d88c1d54e01d824e13f11729128570f029931e4038c30"
Jul 28 21:12:05.780783 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:12:05.780753    2576 scope.go:117] "RemoveContainer" containerID="9b8ee6525359ad3e8773796ac614770ff55b465b5784beb811a47b8b49960b80"
Jul 28 21:12:24.463051 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:12:24.463006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:13:02.754202 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:13:02.754104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:13:18.961482 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:13:18.961391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:13:50.036342 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:13:50.036301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:14:21.255842 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:14:21.255805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:14:22.105981 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:14:22.105932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:14:40.693419 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:14:40.693335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:15:00.132318 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.132283    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"]
Jul 28 21:15:00.135850 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.135831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"
Jul 28 21:15:00.138208 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.138187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cxpsn\""
Jul 28 21:15:00.147668 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.147643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"
Jul 28 21:15:00.148281 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.148259    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"]
Jul 28 21:15:00.288397 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:00.288344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"]
Jul 28 21:15:00.290486 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:15:00.290461    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ced2ad_16a8_4c8d_be87_b82933b778d5.slice/crio-9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b WatchSource:0}: Error finding container 9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b: Status 404 returned error can't find the container with id 9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b
Jul 28 21:15:01.119005 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:01.118972    2576 generic.go:356] "Generic (PLEG): container finished" podID="35ced2ad-16a8-4c8d-be87-b82933b778d5" containerID="eb4d1bb4d3b0b5551063a27489ede4e2e5f18c9b49540119395314da88201b4d" exitCode=0
Jul 28 21:15:01.119207 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:01.119064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk" event={"ID":"35ced2ad-16a8-4c8d-be87-b82933b778d5","Type":"ContainerDied","Data":"eb4d1bb4d3b0b5551063a27489ede4e2e5f18c9b49540119395314da88201b4d"}
Jul 28 21:15:01.119207 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:01.119124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk" event={"ID":"35ced2ad-16a8-4c8d-be87-b82933b778d5","Type":"ContainerStarted","Data":"9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b"}
Jul 28 21:15:02.266794 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:02.266760    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"
Jul 28 21:15:03.131576 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:03.131541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk" event={"ID":"35ced2ad-16a8-4c8d-be87-b82933b778d5","Type":"ContainerDied","Data":"9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b"}
Jul 28 21:15:03.131757 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:03.131588    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a004c75ba894a4c37fa0fa49f7a5ca19ee873d10827bf1d598265d9d6b1594b"
Jul 28 21:15:03.131757 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:03.131547    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"
Jul 28 21:15:12.195236 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:15:12.195201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:34.947898 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:34.947856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:35.807528 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:35.807478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:36.303504 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.303468    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:36.303908 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.303887    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35ced2ad-16a8-4c8d-be87-b82933b778d5" containerName="cleanup"
Jul 28 21:16:36.303908 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.303899    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ced2ad-16a8-4c8d-be87-b82933b778d5" containerName="cleanup"
Jul 28 21:16:36.303992 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.303979    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="35ced2ad-16a8-4c8d-be87-b82933b778d5" containerName="cleanup"
Jul 28 21:16:36.307215 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.307190    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:36.316227 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.316193    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:36.391958 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.391921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxnkd\" (UniqueName: \"kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd\") pod \"kuadrant-operator-catalog-g7hr9\" (UID: \"4311a652-b123-444e-8767-f31b8b8e302b\") " pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:36.493291 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.493251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxnkd\" (UniqueName: \"kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd\") pod \"kuadrant-operator-catalog-g7hr9\" (UID: \"4311a652-b123-444e-8767-f31b8b8e302b\") " pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:36.501995 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.501969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxnkd\" (UniqueName: \"kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd\") pod \"kuadrant-operator-catalog-g7hr9\" (UID: \"4311a652-b123-444e-8767-f31b8b8e302b\") " pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:36.618457 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.618327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:36.764640 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.764607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:36.765830 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:36.765802    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4311a652_b123_444e_8767_f31b8b8e302b.slice/crio-0b08f3c3fdc7332ec616895768ee66f74de790d93704a37c2ebfb88ff2d6849d WatchSource:0}: Error finding container 0b08f3c3fdc7332ec616895768ee66f74de790d93704a37c2ebfb88ff2d6849d: Status 404 returned error can't find the container with id 0b08f3c3fdc7332ec616895768ee66f74de790d93704a37c2ebfb88ff2d6849d
Jul 28 21:16:36.767605 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.767587    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:16:36.814120 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.814079    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513291440"]
Jul 28 21:16:36.818715 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.818692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:36.821239 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.821206    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6cw4n\""
Jul 28 21:16:36.823991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.823965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513291440"]
Jul 28 21:16:36.897121 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.897011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jskf5\" (UniqueName: \"kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5\") pod \"test-curl-7211-140689513291440\" (UID: \"82002cf9-64d5-44ca-a59b-6f6c77edf167\") " pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:36.998660 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:36.998622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jskf5\" (UniqueName: \"kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5\") pod \"test-curl-7211-140689513291440\" (UID: \"82002cf9-64d5-44ca-a59b-6f6c77edf167\") " pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:37.006389 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.006345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jskf5\" (UniqueName: \"kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5\") pod \"test-curl-7211-140689513291440\" (UID: \"82002cf9-64d5-44ca-a59b-6f6c77edf167\") " pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:37.129873 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.129838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:37.275252 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.275224    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513291440"]
Jul 28 21:16:37.278196 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:37.278171    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82002cf9_64d5_44ca_a59b_6f6c77edf167.slice/crio-ddd8083ae2ff3fc9ec12bfbf1bcb02fa8455308ae66ed16900409051725c1019 WatchSource:0}: Error finding container ddd8083ae2ff3fc9ec12bfbf1bcb02fa8455308ae66ed16900409051725c1019: Status 404 returned error can't find the container with id ddd8083ae2ff3fc9ec12bfbf1bcb02fa8455308ae66ed16900409051725c1019
Jul 28 21:16:37.675355 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.675311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513291440" event={"ID":"82002cf9-64d5-44ca-a59b-6f6c77edf167","Type":"ContainerStarted","Data":"ddd8083ae2ff3fc9ec12bfbf1bcb02fa8455308ae66ed16900409051725c1019"}
Jul 28 21:16:37.677226 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.677193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" event={"ID":"4311a652-b123-444e-8767-f31b8b8e302b","Type":"ContainerStarted","Data":"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"}
Jul 28 21:16:37.677409 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.677247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" event={"ID":"4311a652-b123-444e-8767-f31b8b8e302b","Type":"ContainerStarted","Data":"0b08f3c3fdc7332ec616895768ee66f74de790d93704a37c2ebfb88ff2d6849d"}
Jul 28 21:16:37.694249 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:37.694195    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" podStartSLOduration=1.247508982 podStartE2EDuration="1.694179236s" totalImagesPullingTime="446.670254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:16:37.692726872 +0000 UTC m=+3092.877831578" watchObservedRunningTime="2026-07-28 21:16:37.694179236 +0000 UTC m=+3092.879283938"
Jul 28 21:16:38.684650 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:38.684619    2576 generic.go:356] "Generic (PLEG): container finished" podID="82002cf9-64d5-44ca-a59b-6f6c77edf167" containerID="063bfc67d9dc84f9e970d5b693e3c1527f6523684db69bab4e7c43d3578643fe" exitCode=0
Jul 28 21:16:38.685031 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:38.684708    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513291440" event={"ID":"82002cf9-64d5-44ca-a59b-6f6c77edf167","Type":"ContainerDied","Data":"063bfc67d9dc84f9e970d5b693e3c1527f6523684db69bab4e7c43d3578643fe"}
Jul 28 21:16:39.831900 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.831876    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:39.852131 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.852101    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513291440_82002cf9-64d5-44ca-a59b-6f6c77edf167/test-curl-7211-140689513291440/0.log"
Jul 28 21:16:39.877872 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.877844    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513291440"]
Jul 28 21:16:39.881072 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.881044    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513291440"]
Jul 28 21:16:39.925275 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.925245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jskf5\" (UniqueName: \"kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5\") pod \"82002cf9-64d5-44ca-a59b-6f6c77edf167\" (UID: \"82002cf9-64d5-44ca-a59b-6f6c77edf167\") "
Jul 28 21:16:39.927256 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:39.927236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5" (OuterVolumeSpecName: "kube-api-access-jskf5") pod "82002cf9-64d5-44ca-a59b-6f6c77edf167" (UID: "82002cf9-64d5-44ca-a59b-6f6c77edf167"). InnerVolumeSpecName "kube-api-access-jskf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:40.003282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.003249    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513367216"]
Jul 28 21:16:40.003701 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.003687    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82002cf9-64d5-44ca-a59b-6f6c77edf167" containerName="test-curl-7211-140689513291440"
Jul 28 21:16:40.003752 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.003702    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="82002cf9-64d5-44ca-a59b-6f6c77edf167" containerName="test-curl-7211-140689513291440"
Jul 28 21:16:40.003789 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.003760    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="82002cf9-64d5-44ca-a59b-6f6c77edf167" containerName="test-curl-7211-140689513291440"
Jul 28 21:16:40.006975 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.006960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:40.011232 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.011204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513367216"]
Jul 28 21:16:40.026618 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.026588    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jskf5\" (UniqueName: \"kubernetes.io/projected/82002cf9-64d5-44ca-a59b-6f6c77edf167-kube-api-access-jskf5\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:40.127321 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.127283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt24k\" (UniqueName: \"kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k\") pod \"test-curl-7211-140689513367216\" (UID: \"eab9babf-b169-40c0-9b44-fa2a41521019\") " pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:40.227841 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.227797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt24k\" (UniqueName: \"kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k\") pod \"test-curl-7211-140689513367216\" (UID: \"eab9babf-b169-40c0-9b44-fa2a41521019\") " pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:40.235654 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.235618    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt24k\" (UniqueName: \"kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k\") pod \"test-curl-7211-140689513367216\" (UID: \"eab9babf-b169-40c0-9b44-fa2a41521019\") " pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:40.318591 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.318500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:40.461934 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.461906    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513367216"]
Jul 28 21:16:40.463643 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:40.463622    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeab9babf_b169_40c0_9b44_fa2a41521019.slice/crio-12fcd750fbb41731bb48543910a35caafe95db535291c50abe06c293aa1d6318 WatchSource:0}: Error finding container 12fcd750fbb41731bb48543910a35caafe95db535291c50abe06c293aa1d6318: Status 404 returned error can't find the container with id 12fcd750fbb41731bb48543910a35caafe95db535291c50abe06c293aa1d6318
Jul 28 21:16:40.697806 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.697730    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:40.697806 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.697777    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddd8083ae2ff3fc9ec12bfbf1bcb02fa8455308ae66ed16900409051725c1019"
Jul 28 21:16:40.699147 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:40.699124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513367216" event={"ID":"eab9babf-b169-40c0-9b44-fa2a41521019","Type":"ContainerStarted","Data":"12fcd750fbb41731bb48543910a35caafe95db535291c50abe06c293aa1d6318"}
Jul 28 21:16:40.708643 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:16:40.708614    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7211-140689513291440\" is forbidden: User \"system:node:ip-10-0-139-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-131.ec2.internal' and this object" podUID="82002cf9-64d5-44ca-a59b-6f6c77edf167" pod="opendatahub/test-curl-7211-140689513291440"
Jul 28 21:16:41.301001 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:41.300968    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82002cf9-64d5-44ca-a59b-6f6c77edf167" path="/var/lib/kubelet/pods/82002cf9-64d5-44ca-a59b-6f6c77edf167/volumes"
Jul 28 21:16:41.706871 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:41.706772    2576 generic.go:356] "Generic (PLEG): container finished" podID="eab9babf-b169-40c0-9b44-fa2a41521019" containerID="02cec7066e68b16f91df5b9d63a89fd9ef176b454f09f18c60c6eb0e38cd7f8e" exitCode=0
Jul 28 21:16:41.707044 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:41.706870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513367216" event={"ID":"eab9babf-b169-40c0-9b44-fa2a41521019","Type":"ContainerDied","Data":"02cec7066e68b16f91df5b9d63a89fd9ef176b454f09f18c60c6eb0e38cd7f8e"}
Jul 28 21:16:42.851652 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.851623    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:42.871465 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.871437    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513367216_eab9babf-b169-40c0-9b44-fa2a41521019/test-curl-7211-140689513367216/0.log"
Jul 28 21:16:42.898273 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.898246    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513367216"]
Jul 28 21:16:42.901908 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.901883    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513367216"]
Jul 28 21:16:42.952649 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.952618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt24k\" (UniqueName: \"kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k\") pod \"eab9babf-b169-40c0-9b44-fa2a41521019\" (UID: \"eab9babf-b169-40c0-9b44-fa2a41521019\") "
Jul 28 21:16:42.954659 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:42.954640    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k" (OuterVolumeSpecName: "kube-api-access-tt24k") pod "eab9babf-b169-40c0-9b44-fa2a41521019" (UID: "eab9babf-b169-40c0-9b44-fa2a41521019"). InnerVolumeSpecName "kube-api-access-tt24k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:43.053859 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.053829    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt24k\" (UniqueName: \"kubernetes.io/projected/eab9babf-b169-40c0-9b44-fa2a41521019-kube-api-access-tt24k\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:43.064746 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.064714    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:43.065156 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.065143    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab9babf-b169-40c0-9b44-fa2a41521019" containerName="test-curl-7211-140689513367216"
Jul 28 21:16:43.065211 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.065157    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab9babf-b169-40c0-9b44-fa2a41521019" containerName="test-curl-7211-140689513367216"
Jul 28 21:16:43.065252 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.065226    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab9babf-b169-40c0-9b44-fa2a41521019" containerName="test-curl-7211-140689513367216"
Jul 28 21:16:43.068320 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.068305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:43.074289 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.074263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:43.154523 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.154493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp66x\" (UniqueName: \"kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x\") pod \"test-curl-7211-140689513314320\" (UID: \"2dba9843-e15d-41db-9aac-e82c17aebbac\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:43.255839 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.255805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp66x\" (UniqueName: \"kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x\") pod \"test-curl-7211-140689513314320\" (UID: \"2dba9843-e15d-41db-9aac-e82c17aebbac\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:43.264022 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.263998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp66x\" (UniqueName: \"kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x\") pod \"test-curl-7211-140689513314320\" (UID: \"2dba9843-e15d-41db-9aac-e82c17aebbac\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:43.300566 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.300535    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eab9babf-b169-40c0-9b44-fa2a41521019" path="/var/lib/kubelet/pods/eab9babf-b169-40c0-9b44-fa2a41521019/volumes"
Jul 28 21:16:43.379288 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.379252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:43.512118 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.511990    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:43.514698 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:43.514671    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dba9843_e15d_41db_9aac_e82c17aebbac.slice/crio-0ddfdb939f3fc8e839ba473d06b123dd0c3daf02192bc362823ab8437ac8ed7e WatchSource:0}: Error finding container 0ddfdb939f3fc8e839ba473d06b123dd0c3daf02192bc362823ab8437ac8ed7e: Status 404 returned error can't find the container with id 0ddfdb939f3fc8e839ba473d06b123dd0c3daf02192bc362823ab8437ac8ed7e
Jul 28 21:16:43.720759 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.720673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513314320" event={"ID":"2dba9843-e15d-41db-9aac-e82c17aebbac","Type":"ContainerStarted","Data":"0ddfdb939f3fc8e839ba473d06b123dd0c3daf02192bc362823ab8437ac8ed7e"}
Jul 28 21:16:43.721928 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.721904    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513367216"
Jul 28 21:16:43.722053 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:43.721912    2576 scope.go:117] "RemoveContainer" containerID="02cec7066e68b16f91df5b9d63a89fd9ef176b454f09f18c60c6eb0e38cd7f8e"
Jul 28 21:16:44.729188 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:44.729158    2576 generic.go:356] "Generic (PLEG): container finished" podID="2dba9843-e15d-41db-9aac-e82c17aebbac" containerID="a68c7993f75c523b502259b02ed6fd3d24a5fbd7d072351eed4a8c5755cd482b" exitCode=0
Jul 28 21:16:44.729688 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:44.729247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513314320" event={"ID":"2dba9843-e15d-41db-9aac-e82c17aebbac","Type":"ContainerDied","Data":"a68c7993f75c523b502259b02ed6fd3d24a5fbd7d072351eed4a8c5755cd482b"}
Jul 28 21:16:45.893192 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.893169    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:45.918391 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.918048    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513314320_2dba9843-e15d-41db-9aac-e82c17aebbac/test-curl-7211-140689513314320/0.log"
Jul 28 21:16:45.949173 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.949143    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:45.953576 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.953547    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:45.984421 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.984394    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp66x\" (UniqueName: \"kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x\") pod \"2dba9843-e15d-41db-9aac-e82c17aebbac\" (UID: \"2dba9843-e15d-41db-9aac-e82c17aebbac\") "
Jul 28 21:16:45.986529 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:45.986466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x" (OuterVolumeSpecName: "kube-api-access-qp66x") pod "2dba9843-e15d-41db-9aac-e82c17aebbac" (UID: "2dba9843-e15d-41db-9aac-e82c17aebbac"). InnerVolumeSpecName "kube-api-access-qp66x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:46.080759 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.080723    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:46.081171 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.081157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dba9843-e15d-41db-9aac-e82c17aebbac" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:46.081216 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.081174    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dba9843-e15d-41db-9aac-e82c17aebbac" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:46.081252 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.081242    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dba9843-e15d-41db-9aac-e82c17aebbac" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:46.083350 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.083334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.085988 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.085970    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp66x\" (UniqueName: \"kubernetes.io/projected/2dba9843-e15d-41db-9aac-e82c17aebbac-kube-api-access-qp66x\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:46.090956 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.090672    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:46.187196 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.187161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4zth\" (UniqueName: \"kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth\") pod \"test-curl-7211-140689513314320\" (UID: \"cc298808-520c-4262-bf95-9d5a136d1405\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.288113 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.288082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4zth\" (UniqueName: \"kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth\") pod \"test-curl-7211-140689513314320\" (UID: \"cc298808-520c-4262-bf95-9d5a136d1405\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.296545 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.296518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4zth\" (UniqueName: \"kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth\") pod \"test-curl-7211-140689513314320\" (UID: \"cc298808-520c-4262-bf95-9d5a136d1405\") " pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.394133 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.394093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.539393 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.539349    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:46.540949 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:46.540928    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc298808_520c_4262_bf95_9d5a136d1405.slice/crio-fd29f86497223755ea6aab6a112122d2c719f3d48bba17fbc9df83bfccce41e2 WatchSource:0}: Error finding container fd29f86497223755ea6aab6a112122d2c719f3d48bba17fbc9df83bfccce41e2: Status 404 returned error can't find the container with id fd29f86497223755ea6aab6a112122d2c719f3d48bba17fbc9df83bfccce41e2
Jul 28 21:16:46.618990 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.618957    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:46.619171 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.619027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:46.640789 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.640765    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:46.742217 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.742188    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:46.742430 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.742190    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ddfdb939f3fc8e839ba473d06b123dd0c3daf02192bc362823ab8437ac8ed7e"
Jul 28 21:16:46.743474 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.743447    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513314320" event={"ID":"cc298808-520c-4262-bf95-9d5a136d1405","Type":"ContainerStarted","Data":"fd29f86497223755ea6aab6a112122d2c719f3d48bba17fbc9df83bfccce41e2"}
Jul 28 21:16:46.754076 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.754041    2576 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7211-140689513314320" oldPodUID="2dba9843-e15d-41db-9aac-e82c17aebbac" podUID="cc298808-520c-4262-bf95-9d5a136d1405"
Jul 28 21:16:46.765308 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.765287    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:46.872753 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:46.872727    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:47.300864 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:47.300832    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dba9843-e15d-41db-9aac-e82c17aebbac" path="/var/lib/kubelet/pods/2dba9843-e15d-41db-9aac-e82c17aebbac/volumes"
Jul 28 21:16:47.750686 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:47.750649    2576 generic.go:356] "Generic (PLEG): container finished" podID="cc298808-520c-4262-bf95-9d5a136d1405" containerID="e30cf071b3e8fc5059e028d1252d5439621ce5dfb73353ef16573ce49e73616e" exitCode=0
Jul 28 21:16:47.750873 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:47.750736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513314320" event={"ID":"cc298808-520c-4262-bf95-9d5a136d1405","Type":"ContainerDied","Data":"e30cf071b3e8fc5059e028d1252d5439621ce5dfb73353ef16573ce49e73616e"}
Jul 28 21:16:48.757203 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:48.757136    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" podUID="4311a652-b123-444e-8767-f31b8b8e302b" containerName="registry-server" containerID="cri-o://28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6" gracePeriod=2
Jul 28 21:16:48.900677 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:48.900653    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:48.921454 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:48.921423    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513314320_cc298808-520c-4262-bf95-9d5a136d1405/test-curl-7211-140689513314320/0.log"
Jul 28 21:16:48.949424 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:48.949392    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:48.952723 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:48.952697    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513314320"]
Jul 28 21:16:49.011991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.011927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4zth\" (UniqueName: \"kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth\") pod \"cc298808-520c-4262-bf95-9d5a136d1405\" (UID: \"cc298808-520c-4262-bf95-9d5a136d1405\") "
Jul 28 21:16:49.013839 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.013812    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth" (OuterVolumeSpecName: "kube-api-access-f4zth") pod "cc298808-520c-4262-bf95-9d5a136d1405" (UID: "cc298808-520c-4262-bf95-9d5a136d1405"). InnerVolumeSpecName "kube-api-access-f4zth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:49.020173 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.020154    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:49.073760 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.073726    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:49.074159 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074145    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc298808-520c-4262-bf95-9d5a136d1405" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:49.074201 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074161    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc298808-520c-4262-bf95-9d5a136d1405" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:49.074201 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074170    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4311a652-b123-444e-8767-f31b8b8e302b" containerName="registry-server"
Jul 28 21:16:49.074201 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074176    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4311a652-b123-444e-8767-f31b8b8e302b" containerName="registry-server"
Jul 28 21:16:49.074297 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074248    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4311a652-b123-444e-8767-f31b8b8e302b" containerName="registry-server"
Jul 28 21:16:49.074297 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.074257    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc298808-520c-4262-bf95-9d5a136d1405" containerName="test-curl-7211-140689513314320"
Jul 28 21:16:49.076207 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.076194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:49.081188 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.081164    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:49.113056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.113027    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxnkd\" (UniqueName: \"kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd\") pod \"4311a652-b123-444e-8767-f31b8b8e302b\" (UID: \"4311a652-b123-444e-8767-f31b8b8e302b\") "
Jul 28 21:16:49.113295 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.113280    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4zth\" (UniqueName: \"kubernetes.io/projected/cc298808-520c-4262-bf95-9d5a136d1405-kube-api-access-f4zth\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:49.115075 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.115049    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd" (OuterVolumeSpecName: "kube-api-access-vxnkd") pod "4311a652-b123-444e-8767-f31b8b8e302b" (UID: "4311a652-b123-444e-8767-f31b8b8e302b"). InnerVolumeSpecName "kube-api-access-vxnkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:49.213786 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.213748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h44z\" (UniqueName: \"kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z\") pod \"test-curl-7211-140689513315328\" (UID: \"f59d57fc-a465-4c4e-a093-4619a10886e1\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:49.213964 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.213944    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxnkd\" (UniqueName: \"kubernetes.io/projected/4311a652-b123-444e-8767-f31b8b8e302b-kube-api-access-vxnkd\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:49.301820 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.301739    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc298808-520c-4262-bf95-9d5a136d1405" path="/var/lib/kubelet/pods/cc298808-520c-4262-bf95-9d5a136d1405/volumes"
Jul 28 21:16:49.314986 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.314959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h44z\" (UniqueName: \"kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z\") pod \"test-curl-7211-140689513315328\" (UID: \"f59d57fc-a465-4c4e-a093-4619a10886e1\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:49.324343 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.324316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h44z\" (UniqueName: \"kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z\") pod \"test-curl-7211-140689513315328\" (UID: \"f59d57fc-a465-4c4e-a093-4619a10886e1\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:49.387828 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.387797    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:49.523125 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.523094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:49.525627 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:49.525599    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf59d57fc_a465_4c4e_a093_4619a10886e1.slice/crio-df446c65340790a567896af8e2225f28b3d2bbf3086f05803d45f584e6be7c91 WatchSource:0}: Error finding container df446c65340790a567896af8e2225f28b3d2bbf3086f05803d45f584e6be7c91: Status 404 returned error can't find the container with id df446c65340790a567896af8e2225f28b3d2bbf3086f05803d45f584e6be7c91
Jul 28 21:16:49.771218 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.771178    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513314320"
Jul 28 21:16:49.771699 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.771186    2576 scope.go:117] "RemoveContainer" containerID="e30cf071b3e8fc5059e028d1252d5439621ce5dfb73353ef16573ce49e73616e"
Jul 28 21:16:49.772632 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.772568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513315328" event={"ID":"f59d57fc-a465-4c4e-a093-4619a10886e1","Type":"ContainerStarted","Data":"df446c65340790a567896af8e2225f28b3d2bbf3086f05803d45f584e6be7c91"}
Jul 28 21:16:49.773958 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.773933    2576 generic.go:356] "Generic (PLEG): container finished" podID="4311a652-b123-444e-8767-f31b8b8e302b" containerID="28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6" exitCode=0
Jul 28 21:16:49.774055 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.773970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" event={"ID":"4311a652-b123-444e-8767-f31b8b8e302b","Type":"ContainerDied","Data":"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"}
Jul 28 21:16:49.774055 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.773977    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9"
Jul 28 21:16:49.774055 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.774031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-g7hr9" event={"ID":"4311a652-b123-444e-8767-f31b8b8e302b","Type":"ContainerDied","Data":"0b08f3c3fdc7332ec616895768ee66f74de790d93704a37c2ebfb88ff2d6849d"}
Jul 28 21:16:49.787641 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.787621    2576 scope.go:117] "RemoveContainer" containerID="28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"
Jul 28 21:16:49.796682 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.796664    2576 scope.go:117] "RemoveContainer" containerID="28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"
Jul 28 21:16:49.797025 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:16:49.796995    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6\": container with ID starting with 28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6 not found: ID does not exist" containerID="28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"
Jul 28 21:16:49.797128 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.797036    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6"} err="failed to get container status \"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6\": rpc error: code = NotFound desc = could not find container \"28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6\": container with ID starting with 28c27fc0511b14428501282e72a993b63377c9e7e0bfcbfe5d3c00eccccb79e6 not found: ID does not exist"
Jul 28 21:16:49.799849 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.799830    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:49.803863 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:49.803842    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g7hr9"]
Jul 28 21:16:50.782492 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:50.782456    2576 generic.go:356] "Generic (PLEG): container finished" podID="f59d57fc-a465-4c4e-a093-4619a10886e1" containerID="736f2a00a01692084a2ab8b05d76f4664214167b70389606bbf2f61adce27304" exitCode=0
Jul 28 21:16:50.782883 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:50.782549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513315328" event={"ID":"f59d57fc-a465-4c4e-a093-4619a10886e1","Type":"ContainerDied","Data":"736f2a00a01692084a2ab8b05d76f4664214167b70389606bbf2f61adce27304"}
Jul 28 21:16:51.301330 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:51.301294    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4311a652-b123-444e-8767-f31b8b8e302b" path="/var/lib/kubelet/pods/4311a652-b123-444e-8767-f31b8b8e302b/volumes"
Jul 28 21:16:51.929713 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:51.929688    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:51.949869 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:51.949831    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513315328_f59d57fc-a465-4c4e-a093-4619a10886e1/test-curl-7211-140689513315328/0.log"
Jul 28 21:16:51.975438 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:51.975399    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:51.979506 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:51.979479    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:52.039592 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.039502    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h44z\" (UniqueName: \"kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z\") pod \"f59d57fc-a465-4c4e-a093-4619a10886e1\" (UID: \"f59d57fc-a465-4c4e-a093-4619a10886e1\") "
Jul 28 21:16:52.041594 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.041569    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z" (OuterVolumeSpecName: "kube-api-access-4h44z") pod "f59d57fc-a465-4c4e-a093-4619a10886e1" (UID: "f59d57fc-a465-4c4e-a093-4619a10886e1"). InnerVolumeSpecName "kube-api-access-4h44z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:52.098623 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.098579    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:52.099180 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.099161    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f59d57fc-a465-4c4e-a093-4619a10886e1" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:52.099287 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.099183    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f59d57fc-a465-4c4e-a093-4619a10886e1" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:52.099347 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.099329    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f59d57fc-a465-4c4e-a093-4619a10886e1" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:52.101667 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.101644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.108764 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.108732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:52.140887 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.140846    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h44z\" (UniqueName: \"kubernetes.io/projected/f59d57fc-a465-4c4e-a093-4619a10886e1-kube-api-access-4h44z\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:52.242292 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.242256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqqc6\" (UniqueName: \"kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6\") pod \"test-curl-7211-140689513315328\" (UID: \"ff39e089-6cf0-41ef-8bf6-131add96dbdc\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.343067 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.342969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqqc6\" (UniqueName: \"kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6\") pod \"test-curl-7211-140689513315328\" (UID: \"ff39e089-6cf0-41ef-8bf6-131add96dbdc\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.351568 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.351535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqqc6\" (UniqueName: \"kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6\") pod \"test-curl-7211-140689513315328\" (UID: \"ff39e089-6cf0-41ef-8bf6-131add96dbdc\") " pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.413295 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.413258    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.564460 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.564432    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:52.566239 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:16:52.566206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff39e089_6cf0_41ef_8bf6_131add96dbdc.slice/crio-2a7cf7fdd41fda25105d52d4fb187cc578dcfe0077289110c8d451ec6b11937c WatchSource:0}: Error finding container 2a7cf7fdd41fda25105d52d4fb187cc578dcfe0077289110c8d451ec6b11937c: Status 404 returned error can't find the container with id 2a7cf7fdd41fda25105d52d4fb187cc578dcfe0077289110c8d451ec6b11937c
Jul 28 21:16:52.796598 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.796566    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:52.796796 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.796604    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df446c65340790a567896af8e2225f28b3d2bbf3086f05803d45f584e6be7c91"
Jul 28 21:16:52.797867 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.797843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513315328" event={"ID":"ff39e089-6cf0-41ef-8bf6-131add96dbdc","Type":"ContainerStarted","Data":"2a7cf7fdd41fda25105d52d4fb187cc578dcfe0077289110c8d451ec6b11937c"}
Jul 28 21:16:52.809278 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:52.809242    2576 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7211-140689513315328" oldPodUID="f59d57fc-a465-4c4e-a093-4619a10886e1" podUID="ff39e089-6cf0-41ef-8bf6-131add96dbdc"
Jul 28 21:16:53.300785 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:53.300750    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f59d57fc-a465-4c4e-a093-4619a10886e1" path="/var/lib/kubelet/pods/f59d57fc-a465-4c4e-a093-4619a10886e1/volumes"
Jul 28 21:16:53.812434 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:53.812398    2576 generic.go:356] "Generic (PLEG): container finished" podID="ff39e089-6cf0-41ef-8bf6-131add96dbdc" containerID="f36e0c2e3b7094f6efa9ba8f72a5d60f25fd874ee46815b1f24c2342560101df" exitCode=0
Jul 28 21:16:53.812627 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:53.812528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7211-140689513315328" event={"ID":"ff39e089-6cf0-41ef-8bf6-131add96dbdc","Type":"ContainerDied","Data":"f36e0c2e3b7094f6efa9ba8f72a5d60f25fd874ee46815b1f24c2342560101df"}
Jul 28 21:16:54.966554 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:54.966532    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:54.986520 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:54.986490    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7211-140689513315328_ff39e089-6cf0-41ef-8bf6-131add96dbdc/test-curl-7211-140689513315328/0.log"
Jul 28 21:16:55.015149 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.015118    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:55.018902 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.018877    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7211-140689513315328"]
Jul 28 21:16:55.068229 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.068199    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqqc6\" (UniqueName: \"kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6\") pod \"ff39e089-6cf0-41ef-8bf6-131add96dbdc\" (UID: \"ff39e089-6cf0-41ef-8bf6-131add96dbdc\") "
Jul 28 21:16:55.070311 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.070258    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6" (OuterVolumeSpecName: "kube-api-access-fqqc6") pod "ff39e089-6cf0-41ef-8bf6-131add96dbdc" (UID: "ff39e089-6cf0-41ef-8bf6-131add96dbdc"). InnerVolumeSpecName "kube-api-access-fqqc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:16:55.169881 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.169843    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqqc6\" (UniqueName: \"kubernetes.io/projected/ff39e089-6cf0-41ef-8bf6-131add96dbdc-kube-api-access-fqqc6\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:16:55.301327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.301289    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff39e089-6cf0-41ef-8bf6-131add96dbdc" path="/var/lib/kubelet/pods/ff39e089-6cf0-41ef-8bf6-131add96dbdc/volumes"
Jul 28 21:16:55.542082 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.542055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:55.827162 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.827075    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7211-140689513315328"
Jul 28 21:16:55.827162 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.827150    2576 scope.go:117] "RemoveContainer" containerID="f36e0c2e3b7094f6efa9ba8f72a5d60f25fd874ee46815b1f24c2342560101df"
Jul 28 21:16:55.971644 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:55.971613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:56.072475 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:56.072437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:58.947465 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:58.947425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:59.045062 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.045016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:59.209463 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.209357    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:16:59.209920 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.209900    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff39e089-6cf0-41ef-8bf6-131add96dbdc" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:59.209920 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.209920    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff39e089-6cf0-41ef-8bf6-131add96dbdc" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:59.210062 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.210000    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff39e089-6cf0-41ef-8bf6-131add96dbdc" containerName="test-curl-7211-140689513315328"
Jul 28 21:16:59.213299 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.213274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.216042 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.216020    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d6ef24-serving-cert\""
Jul 28 21:16:59.230356 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.230314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:16:59.256301 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.256244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:59.301915 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.301878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.302108 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.301930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf94r\" (UniqueName: \"kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.302108 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.301996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.402429 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.402395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.402636 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.402496    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.402636 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.402541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf94r\" (UniqueName: \"kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.402760 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:16:59.402678    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-serving-cert: secret "maas-api-e2e-ipp-a-d6ef24-serving-cert" not found
Jul 28 21:16:59.402760 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:16:59.402746    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls podName:ae8f2970-6958-4825-a237-90166f7c523d nodeName:}" failed. No retries permitted until 2026-07-28 21:16:59.902728377 +0000 UTC m=+3115.087833061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls") pod "maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" (UID: "ae8f2970-6958-4825-a237-90166f7c523d") : secret "maas-api-e2e-ipp-a-d6ef24-serving-cert" not found
Jul 28 21:16:59.403127 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.403103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.411448 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.411424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf94r\" (UniqueName: \"kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.874901 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.874869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:16:59.906048 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.905988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.909184 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.909159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:16:59.949682 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:16:59.949592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:00.128315 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.128226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:17:00.274494 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.274469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:17:00.276883 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:17:00.276852    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8f2970_6958_4825_a237_90166f7c523d.slice/crio-de8064edaa90b521b69b21e80c76d331394fdaa5de57a21ee8d14a0d47cbae49 WatchSource:0}: Error finding container de8064edaa90b521b69b21e80c76d331394fdaa5de57a21ee8d14a0d47cbae49: Status 404 returned error can't find the container with id de8064edaa90b521b69b21e80c76d331394fdaa5de57a21ee8d14a0d47cbae49
Jul 28 21:17:00.859112 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.859082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" event={"ID":"ae8f2970-6958-4825-a237-90166f7c523d","Type":"ContainerStarted","Data":"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"}
Jul 28 21:17:00.859282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.859140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" event={"ID":"ae8f2970-6958-4825-a237-90166f7c523d","Type":"ContainerStarted","Data":"de8064edaa90b521b69b21e80c76d331394fdaa5de57a21ee8d14a0d47cbae49"}
Jul 28 21:17:00.859282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.859174    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:17:00.877816 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:00.877767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" podStartSLOduration=1.454486266 podStartE2EDuration="1.877753022s" totalImagesPullingTime="423.266756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:17:00.874922135 +0000 UTC m=+3116.060026848" watchObservedRunningTime="2026-07-28 21:17:00.877753022 +0000 UTC m=+3116.062857728"
Jul 28 21:17:06.870549 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:06.870517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:17:24.031112 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:24.031072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:24.728326 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:24.728286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:25.058579 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:25.058528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:25.221946 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:25.221907    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:25.698775 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:25.698737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:25.796962 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:25.796928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.093481 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.093403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.194584 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.194550    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:17:28.197817 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.197783    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.198466 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.198448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.200850 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.200823    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-590ada-serving-cert\""
Jul 28 21:17:28.209735 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.209701    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:17:28.270325 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.270289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.272474 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.272446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wsd6\" (UniqueName: \"kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.272616 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.272509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.272616 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.272548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.373389 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.373274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wsd6\" (UniqueName: \"kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.373389 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.373329    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.373389 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.373348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.373682 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:17:28.373448    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-serving-cert: secret "maas-api-e2e-ipp-b-590ada-serving-cert" not found
Jul 28 21:17:28.373682 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:17:28.373514    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls podName:a607b0c7-d737-4b14-8c70-3b476b96f16e nodeName:}" failed. No retries permitted until 2026-07-28 21:17:28.873496389 +0000 UTC m=+3144.058601073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls") pod "maas-api-e2e-ipp-b-590ada-694bb9b-5587r" (UID: "a607b0c7-d737-4b14-8c70-3b476b96f16e") : secret "maas-api-e2e-ipp-b-590ada-serving-cert" not found
Jul 28 21:17:28.373971 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.373953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.382003 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.381970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wsd6\" (UniqueName: \"kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.710002 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.709909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.856243 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.856211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.877545 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.877516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.880396 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.880356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-590ada-694bb9b-5587r\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:28.953001 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.952963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:28.987636 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.987558    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:17:28.995068 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.992195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:28.995814 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.995788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-590ada-dockercfg-kw8nx\""
Jul 28 21:17:28.996172 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:28.995908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-590ada\""
Jul 28 21:17:29.005327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.005302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:17:29.079066 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.079021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.079289 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.079184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lrj\" (UniqueName: \"kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.112024 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.111996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:29.180643 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.180605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lrj\" (UniqueName: \"kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.180820 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.180702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.181543 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.181521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.191783 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.191754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lrj\" (UniqueName: \"kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj\") pod \"payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.266534 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.266501    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:17:29.268132 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:17:29.268106    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda607b0c7_d737_4b14_8c70_3b476b96f16e.slice/crio-1b56be6f0450112e785189913f289746a6021c546d0e3689f09a099919ba1d39 WatchSource:0}: Error finding container 1b56be6f0450112e785189913f289746a6021c546d0e3689f09a099919ba1d39: Status 404 returned error can't find the container with id 1b56be6f0450112e785189913f289746a6021c546d0e3689f09a099919ba1d39
Jul 28 21:17:29.306351 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.306319    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:29.449997 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:29.449966    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:17:29.451427 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:17:29.451403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bba1b34_7d74_4b5e_97dd_58893340d479.slice/crio-abfa0c13a30d347c47a85eccabfc44b2e84b93243a231a2d959fd2f246f7fbda WatchSource:0}: Error finding container abfa0c13a30d347c47a85eccabfc44b2e84b93243a231a2d959fd2f246f7fbda: Status 404 returned error can't find the container with id abfa0c13a30d347c47a85eccabfc44b2e84b93243a231a2d959fd2f246f7fbda
Jul 28 21:17:30.031610 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.031572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" event={"ID":"9bba1b34-7d74-4b5e-97dd-58893340d479","Type":"ContainerStarted","Data":"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"}
Jul 28 21:17:30.031818 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.031643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" event={"ID":"9bba1b34-7d74-4b5e-97dd-58893340d479","Type":"ContainerStarted","Data":"abfa0c13a30d347c47a85eccabfc44b2e84b93243a231a2d959fd2f246f7fbda"}
Jul 28 21:17:30.031818 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.031678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:30.033141 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.033119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" event={"ID":"a607b0c7-d737-4b14-8c70-3b476b96f16e","Type":"ContainerStarted","Data":"295906ce989752b26577415efd14c6c80bc55087b1067dc1589435af9a7ef033"}
Jul 28 21:17:30.033220 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.033159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" event={"ID":"a607b0c7-d737-4b14-8c70-3b476b96f16e","Type":"ContainerStarted","Data":"1b56be6f0450112e785189913f289746a6021c546d0e3689f09a099919ba1d39"}
Jul 28 21:17:30.033220 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.033204    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:30.051175 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.051119    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" podStartSLOduration=2.051104652 podStartE2EDuration="2.051104652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:17:30.047890101 +0000 UTC m=+3145.232994808" watchObservedRunningTime="2026-07-28 21:17:30.051104652 +0000 UTC m=+3145.236209358"
Jul 28 21:17:30.066577 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:30.066520    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" podStartSLOduration=1.634124409 podStartE2EDuration="2.066504276s" totalImagesPullingTime="432.379867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:17:30.064124084 +0000 UTC m=+3145.249228812" watchObservedRunningTime="2026-07-28 21:17:30.066504276 +0000 UTC m=+3145.251608983"
Jul 28 21:17:36.045184 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:36.045146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:17:41.042121 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:41.042083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:17:46.156604 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:46.156567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:17:53.201303 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:17:53.201269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:00.367477 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:00.367444    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn_9bba1b34-7d74-4b5e-97dd-58893340d479/payload-processing/0.log"
Jul 28 21:18:05.444219 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:05.444183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:47.666869 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:47.666841    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn_9bba1b34-7d74-4b5e-97dd-58893340d479/payload-processing/0.log"
Jul 28 21:18:47.993816 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:47.993777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:48.399503 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:48.399409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:48.758830 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:48.758800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:51.562396 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.562340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:51.656900 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.656870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:51.662596 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.662565    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:18:51.666166 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.666146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.670289 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.670267    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8951c6-serving-cert\""
Jul 28 21:18:51.680747 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.680724    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:18:51.753904 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.753872    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.754086 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.753917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spnb7\" (UniqueName: \"kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.754086 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.754053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.757163 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.757137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:51.854991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.854896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.854991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.854952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.855191 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.854998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spnb7\" (UniqueName: \"kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.855191 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:18:51.855065    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-serving-cert: secret "maas-api-e2e-ipp-cleanup-8951c6-serving-cert" not found
Jul 28 21:18:51.855191 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:18:51.855128    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls podName:15d537f5-cf4b-458b-83e9-e68cd86e8940 nodeName:}" failed. No retries permitted until 2026-07-28 21:18:52.355106542 +0000 UTC m=+3227.540211238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" (UID: "15d537f5-cf4b-458b-83e9-e68cd86e8940") : secret "maas-api-e2e-ipp-cleanup-8951c6-serving-cert" not found
Jul 28 21:18:51.855623 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.855603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:51.865279 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:51.865247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spnb7\" (UniqueName: \"kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:52.031821 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.031789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:52.360625 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.360590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:52.363299 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.363270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:52.400032 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.399997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:52.486122 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.486076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:18:52.536468 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.536428    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:18:52.542332 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.542301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.549527 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.549500    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8951c6-dockercfg-r97qp\""
Jul 28 21:18:52.549979 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.549960    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8951c6\""
Jul 28 21:18:52.551327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.551297    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:18:52.576779 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.576745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:52.663626 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.663594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.663796 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.663662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwl25\" (UniqueName: \"kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.735267 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.735215    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:18:52.737334 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:18:52.737305    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d537f5_cf4b_458b_83e9_e68cd86e8940.slice/crio-5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c WatchSource:0}: Error finding container 5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c: Status 404 returned error can't find the container with id 5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c
Jul 28 21:18:52.764916 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.764887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.765054 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.764936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwl25\" (UniqueName: \"kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.765633 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.765609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.775083 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.775063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwl25\" (UniqueName: \"kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25\") pod \"payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:52.860747 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:52.860714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:53.013501 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.013461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:18:53.015023 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:18:53.014988    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2975c36a_2fcc_4797_b6fd_0738f45df246.slice/crio-a6970468b8a5f20924ba718d2e2dd4feb59311b6f507e139fff2397dc8f87cf3 WatchSource:0}: Error finding container a6970468b8a5f20924ba718d2e2dd4feb59311b6f507e139fff2397dc8f87cf3: Status 404 returned error can't find the container with id a6970468b8a5f20924ba718d2e2dd4feb59311b6f507e139fff2397dc8f87cf3
Jul 28 21:18:53.535666 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.535555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" event={"ID":"15d537f5-cf4b-458b-83e9-e68cd86e8940","Type":"ContainerStarted","Data":"4d77933d2afab6ab0e4bd443d1a6b3ac21e7fbe75e91487abecddc870f12e708"}
Jul 28 21:18:53.535666 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.535629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" event={"ID":"15d537f5-cf4b-458b-83e9-e68cd86e8940","Type":"ContainerStarted","Data":"5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c"}
Jul 28 21:18:53.535666 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.535659    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:18:53.537186 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.537156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" event={"ID":"2975c36a-2fcc-4797-b6fd-0738f45df246","Type":"ContainerStarted","Data":"938169f7cf471479352be474783e2ab63aaaf63433a6d29ae7ddc95b98df1b1e"}
Jul 28 21:18:53.537326 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.537207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" event={"ID":"2975c36a-2fcc-4797-b6fd-0738f45df246","Type":"ContainerStarted","Data":"a6970468b8a5f20924ba718d2e2dd4feb59311b6f507e139fff2397dc8f87cf3"}
Jul 28 21:18:53.537326 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.537257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:18:53.557304 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.557256    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" podStartSLOduration=2.041140076 podStartE2EDuration="2.557240902s" totalImagesPullingTime="516.100826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:18:53.555538185 +0000 UTC m=+3228.740642892" watchObservedRunningTime="2026-07-28 21:18:53.557240902 +0000 UTC m=+3228.742345608"
Jul 28 21:18:53.591227 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:53.591169    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" podStartSLOduration=1.591155264 podStartE2EDuration="1.591155264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:18:53.584233184 +0000 UTC m=+3228.769337890" watchObservedRunningTime="2026-07-28 21:18:53.591155264 +0000 UTC m=+3228.776259969"
Jul 28 21:18:59.553295 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:18:59.553213    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:19:04.549548 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:04.549512    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:19:16.637340 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:16.637307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:22.415882 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.415847    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:19:22.416442 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.416060    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" podUID="15d537f5-cf4b-458b-83e9-e68cd86e8940" containerName="maas-api" containerID="cri-o://4d77933d2afab6ab0e4bd443d1a6b3ac21e7fbe75e91487abecddc870f12e708" gracePeriod=30
Jul 28 21:19:22.480837 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.480801    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:19:22.481104 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.481051    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" podUID="2975c36a-2fcc-4797-b6fd-0738f45df246" containerName="payload-processing" containerID="cri-o://938169f7cf471479352be474783e2ab63aaaf63433a6d29ae7ddc95b98df1b1e" gracePeriod=30
Jul 28 21:19:22.725014 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.724986    2576 generic.go:356] "Generic (PLEG): container finished" podID="15d537f5-cf4b-458b-83e9-e68cd86e8940" containerID="4d77933d2afab6ab0e4bd443d1a6b3ac21e7fbe75e91487abecddc870f12e708" exitCode=0
Jul 28 21:19:22.725156 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.725062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" event={"ID":"15d537f5-cf4b-458b-83e9-e68cd86e8940","Type":"ContainerDied","Data":"4d77933d2afab6ab0e4bd443d1a6b3ac21e7fbe75e91487abecddc870f12e708"}
Jul 28 21:19:22.725156 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.725129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n" event={"ID":"15d537f5-cf4b-458b-83e9-e68cd86e8940","Type":"ContainerDied","Data":"5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c"}
Jul 28 21:19:22.725280 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.725158    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e805988641cbfb30cda92bd1e71ca16b61738fdaf2b54b70a979ef21aa0ee5c"
Jul 28 21:19:22.726794 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.726770    2576 generic.go:356] "Generic (PLEG): container finished" podID="2975c36a-2fcc-4797-b6fd-0738f45df246" containerID="938169f7cf471479352be474783e2ab63aaaf63433a6d29ae7ddc95b98df1b1e" exitCode=0
Jul 28 21:19:22.726907 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.726837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" event={"ID":"2975c36a-2fcc-4797-b6fd-0738f45df246","Type":"ContainerDied","Data":"938169f7cf471479352be474783e2ab63aaaf63433a6d29ae7ddc95b98df1b1e"}
Jul 28 21:19:22.735639 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.735617    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:19:22.766611 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.766591    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:19:22.859478 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859443    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") pod \"15d537f5-cf4b-458b-83e9-e68cd86e8940\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") "
Jul 28 21:19:22.859478 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859480    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwl25\" (UniqueName: \"kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25\") pod \"2975c36a-2fcc-4797-b6fd-0738f45df246\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") "
Jul 28 21:19:22.859697 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859529    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume\") pod \"2975c36a-2fcc-4797-b6fd-0738f45df246\" (UID: \"2975c36a-2fcc-4797-b6fd-0738f45df246\") "
Jul 28 21:19:22.859738 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca\") pod \"15d537f5-cf4b-458b-83e9-e68cd86e8940\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") "
Jul 28 21:19:22.859805 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spnb7\" (UniqueName: \"kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7\") pod \"15d537f5-cf4b-458b-83e9-e68cd86e8940\" (UID: \"15d537f5-cf4b-458b-83e9-e68cd86e8940\") "
Jul 28 21:19:22.859950 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.859925    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2975c36a-2fcc-4797-b6fd-0738f45df246" (UID: "2975c36a-2fcc-4797-b6fd-0738f45df246"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:19:22.860084 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.860061    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15d537f5-cf4b-458b-83e9-e68cd86e8940" (UID: "15d537f5-cf4b-458b-83e9-e68cd86e8940"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:19:22.860169 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.860153    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2975c36a-2fcc-4797-b6fd-0738f45df246-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:22.860223 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.860174    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d537f5-cf4b-458b-83e9-e68cd86e8940-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:22.861673 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.861648    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25" (OuterVolumeSpecName: "kube-api-access-pwl25") pod "2975c36a-2fcc-4797-b6fd-0738f45df246" (UID: "2975c36a-2fcc-4797-b6fd-0738f45df246"). InnerVolumeSpecName "kube-api-access-pwl25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:19:22.861766 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.861681    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15d537f5-cf4b-458b-83e9-e68cd86e8940" (UID: "15d537f5-cf4b-458b-83e9-e68cd86e8940"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:19:22.861766 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.861687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7" (OuterVolumeSpecName: "kube-api-access-spnb7") pod "15d537f5-cf4b-458b-83e9-e68cd86e8940" (UID: "15d537f5-cf4b-458b-83e9-e68cd86e8940"). InnerVolumeSpecName "kube-api-access-spnb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:19:22.961056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.960971    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spnb7\" (UniqueName: \"kubernetes.io/projected/15d537f5-cf4b-458b-83e9-e68cd86e8940-kube-api-access-spnb7\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:22.961056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.961003    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d537f5-cf4b-458b-83e9-e68cd86e8940-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:22.961056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:22.961015    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwl25\" (UniqueName: \"kubernetes.io/projected/2975c36a-2fcc-4797-b6fd-0738f45df246-kube-api-access-pwl25\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:23.128743 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.128704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:23.196946 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.196913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:23.734599 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.734518    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"
Jul 28 21:19:23.734599 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.734562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s" event={"ID":"2975c36a-2fcc-4797-b6fd-0738f45df246","Type":"ContainerDied","Data":"a6970468b8a5f20924ba718d2e2dd4feb59311b6f507e139fff2397dc8f87cf3"}
Jul 28 21:19:23.735124 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.734621    2576 scope.go:117] "RemoveContainer" containerID="938169f7cf471479352be474783e2ab63aaaf63433a6d29ae7ddc95b98df1b1e"
Jul 28 21:19:23.735124 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.734737    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"
Jul 28 21:19:23.759661 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.759632    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:19:23.765705 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.765678    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8951c6-855c88d4b4-xbf7n"]
Jul 28 21:19:23.779467 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.779421    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:19:23.783401 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:23.783353    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8951c6-6c7466fdd4-twv7s"]
Jul 28 21:19:25.301499 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:25.301465    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d537f5-cf4b-458b-83e9-e68cd86e8940" path="/var/lib/kubelet/pods/15d537f5-cf4b-458b-83e9-e68cd86e8940/volumes"
Jul 28 21:19:25.301881 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:25.301811    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2975c36a-2fcc-4797-b6fd-0738f45df246" path="/var/lib/kubelet/pods/2975c36a-2fcc-4797-b6fd-0738f45df246/volumes"
Jul 28 21:19:29.379155 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:29.379116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:29.453457 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:29.453426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:29.691948 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:29.691857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:29.963067 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:29.962980    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:49.361278 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.361229    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:19:49.361806 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.361532    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" podUID="ae8f2970-6958-4825-a237-90166f7c523d" containerName="maas-api" containerID="cri-o://28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac" gracePeriod=30
Jul 28 21:19:49.402118 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.402089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:49.626891 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.626824    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:19:49.812965 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.812928    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") pod \"ae8f2970-6958-4825-a237-90166f7c523d\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") "
Jul 28 21:19:49.813178 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.813050    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca\") pod \"ae8f2970-6958-4825-a237-90166f7c523d\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") "
Jul 28 21:19:49.813178 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.813077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf94r\" (UniqueName: \"kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r\") pod \"ae8f2970-6958-4825-a237-90166f7c523d\" (UID: \"ae8f2970-6958-4825-a237-90166f7c523d\") "
Jul 28 21:19:49.813516 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.813487    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae8f2970-6958-4825-a237-90166f7c523d" (UID: "ae8f2970-6958-4825-a237-90166f7c523d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:19:49.814991 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.814958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae8f2970-6958-4825-a237-90166f7c523d" (UID: "ae8f2970-6958-4825-a237-90166f7c523d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:19:49.815174 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.815150    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r" (OuterVolumeSpecName: "kube-api-access-nf94r") pod "ae8f2970-6958-4825-a237-90166f7c523d" (UID: "ae8f2970-6958-4825-a237-90166f7c523d"). InnerVolumeSpecName "kube-api-access-nf94r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:19:49.896333 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.896238    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae8f2970-6958-4825-a237-90166f7c523d" containerID="28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac" exitCode=0
Jul 28 21:19:49.897234 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.896689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"
Jul 28 21:19:49.897234 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.896675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" event={"ID":"ae8f2970-6958-4825-a237-90166f7c523d","Type":"ContainerDied","Data":"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"}
Jul 28 21:19:49.897234 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.897221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5" event={"ID":"ae8f2970-6958-4825-a237-90166f7c523d","Type":"ContainerDied","Data":"de8064edaa90b521b69b21e80c76d331394fdaa5de57a21ee8d14a0d47cbae49"}
Jul 28 21:19:49.897546 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.897256    2576 scope.go:117] "RemoveContainer" containerID="28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"
Jul 28 21:19:49.908516 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.908497    2576 scope.go:117] "RemoveContainer" containerID="28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"
Jul 28 21:19:49.908781 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:19:49.908762    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac\": container with ID starting with 28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac not found: ID does not exist" containerID="28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"
Jul 28 21:19:49.908847 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.908795    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac"} err="failed to get container status \"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac\": rpc error: code = NotFound desc = could not find container \"28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac\": container with ID starting with 28b6194821f28abf5aafeefaec68a414710154ccc908417becf0d6997dd91fac not found: ID does not exist"
Jul 28 21:19:49.914203 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.914181    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8f2970-6958-4825-a237-90166f7c523d-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:49.914203 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.914203    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8f2970-6958-4825-a237-90166f7c523d-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:49.914328 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.914213    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf94r\" (UniqueName: \"kubernetes.io/projected/ae8f2970-6958-4825-a237-90166f7c523d-kube-api-access-nf94r\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:19:49.925077 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.925050    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:19:49.935709 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:49.934962    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d6ef24-6b8bf745c4-824d5"]
Jul 28 21:19:50.666127 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:50.666089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:19:51.301444 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:51.301411    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8f2970-6958-4825-a237-90166f7c523d" path="/var/lib/kubelet/pods/ae8f2970-6958-4825-a237-90166f7c523d/volumes"
Jul 28 21:19:55.148723 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:19:55.148687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:13.592442 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:13.592350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:32.395762 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:32.395682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:32.529711 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:32.529679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:38.818047 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:38.818010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:42.036109 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:42.036071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:42.137135 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:42.137098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:53.551306 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:53.551266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:53.656318 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:53.656286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:53.876396 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:53.876297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:20:54.132511 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:20:54.132430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:05.119601 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.119561    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:21:05.120186 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.119865    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" podUID="a607b0c7-d737-4b14-8c70-3b476b96f16e" containerName="maas-api" containerID="cri-o://295906ce989752b26577415efd14c6c80bc55087b1067dc1589435af9a7ef033" gracePeriod=30
Jul 28 21:21:05.248689 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.248649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:05.286937 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.286899    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:21:05.287148 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.287121    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" podUID="9bba1b34-7d74-4b5e-97dd-58893340d479" containerName="payload-processing" containerID="cri-o://f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d" gracePeriod=30
Jul 28 21:21:05.352932 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.352838    2576 generic.go:356] "Generic (PLEG): container finished" podID="a607b0c7-d737-4b14-8c70-3b476b96f16e" containerID="295906ce989752b26577415efd14c6c80bc55087b1067dc1589435af9a7ef033" exitCode=0
Jul 28 21:21:05.352932 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.352921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" event={"ID":"a607b0c7-d737-4b14-8c70-3b476b96f16e","Type":"ContainerDied","Data":"295906ce989752b26577415efd14c6c80bc55087b1067dc1589435af9a7ef033"}
Jul 28 21:21:05.407065 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.407024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:05.415510 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.415482    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:21:05.507782 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.507753    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") pod \"a607b0c7-d737-4b14-8c70-3b476b96f16e\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") "
Jul 28 21:21:05.507936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.507818    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wsd6\" (UniqueName: \"kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6\") pod \"a607b0c7-d737-4b14-8c70-3b476b96f16e\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") "
Jul 28 21:21:05.507936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.507886    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca\") pod \"a607b0c7-d737-4b14-8c70-3b476b96f16e\" (UID: \"a607b0c7-d737-4b14-8c70-3b476b96f16e\") "
Jul 28 21:21:05.508265 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.508240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a607b0c7-d737-4b14-8c70-3b476b96f16e" (UID: "a607b0c7-d737-4b14-8c70-3b476b96f16e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:21:05.510143 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.510113    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a607b0c7-d737-4b14-8c70-3b476b96f16e" (UID: "a607b0c7-d737-4b14-8c70-3b476b96f16e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:21:05.510249 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.510184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6" (OuterVolumeSpecName: "kube-api-access-6wsd6") pod "a607b0c7-d737-4b14-8c70-3b476b96f16e" (UID: "a607b0c7-d737-4b14-8c70-3b476b96f16e"). InnerVolumeSpecName "kube-api-access-6wsd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:21:05.511562 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.511541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:05.549141 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.549118    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:21:05.608529 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608501    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume\") pod \"9bba1b34-7d74-4b5e-97dd-58893340d479\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") "
Jul 28 21:21:05.608687 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9lrj\" (UniqueName: \"kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj\") pod \"9bba1b34-7d74-4b5e-97dd-58893340d479\" (UID: \"9bba1b34-7d74-4b5e-97dd-58893340d479\") "
Jul 28 21:21:05.608857 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608838    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wsd6\" (UniqueName: \"kubernetes.io/projected/a607b0c7-d737-4b14-8c70-3b476b96f16e-kube-api-access-6wsd6\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:05.608900 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608865    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a607b0c7-d737-4b14-8c70-3b476b96f16e-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:05.608900 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608882    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a607b0c7-d737-4b14-8c70-3b476b96f16e-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:05.608964 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.608892    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bba1b34-7d74-4b5e-97dd-58893340d479" (UID: "9bba1b34-7d74-4b5e-97dd-58893340d479"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:21:05.610542 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.610526    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj" (OuterVolumeSpecName: "kube-api-access-p9lrj") pod "9bba1b34-7d74-4b5e-97dd-58893340d479" (UID: "9bba1b34-7d74-4b5e-97dd-58893340d479"). InnerVolumeSpecName "kube-api-access-p9lrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:21:05.616534 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.616502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:05.709535 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.709455    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bba1b34-7d74-4b5e-97dd-58893340d479-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:05.709535 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:05.709482    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9lrj\" (UniqueName: \"kubernetes.io/projected/9bba1b34-7d74-4b5e-97dd-58893340d479-kube-api-access-p9lrj\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:06.024382 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.024344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:06.360398 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.360274    2576 generic.go:356] "Generic (PLEG): container finished" podID="9bba1b34-7d74-4b5e-97dd-58893340d479" containerID="f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d" exitCode=0
Jul 28 21:21:06.360398 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.360321    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"
Jul 28 21:21:06.360398 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.360358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" event={"ID":"9bba1b34-7d74-4b5e-97dd-58893340d479","Type":"ContainerDied","Data":"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"}
Jul 28 21:21:06.360963 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.360439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn" event={"ID":"9bba1b34-7d74-4b5e-97dd-58893340d479","Type":"ContainerDied","Data":"abfa0c13a30d347c47a85eccabfc44b2e84b93243a231a2d959fd2f246f7fbda"}
Jul 28 21:21:06.360963 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.360472    2576 scope.go:117] "RemoveContainer" containerID="f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"
Jul 28 21:21:06.361916 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.361894    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"
Jul 28 21:21:06.362013 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.361913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r" event={"ID":"a607b0c7-d737-4b14-8c70-3b476b96f16e","Type":"ContainerDied","Data":"1b56be6f0450112e785189913f289746a6021c546d0e3689f09a099919ba1d39"}
Jul 28 21:21:06.371335 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.371313    2576 scope.go:117] "RemoveContainer" containerID="f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"
Jul 28 21:21:06.371683 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:21:06.371660    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d\": container with ID starting with f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d not found: ID does not exist" containerID="f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"
Jul 28 21:21:06.371781 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.371691    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d"} err="failed to get container status \"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d\": rpc error: code = NotFound desc = could not find container \"f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d\": container with ID starting with f23b6b78ee48e86957294ba5df4182bdfc286cc31784f2dc1cb131037cbe751d not found: ID does not exist"
Jul 28 21:21:06.371781 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.371716    2576 scope.go:117] "RemoveContainer" containerID="295906ce989752b26577415efd14c6c80bc55087b1067dc1589435af9a7ef033"
Jul 28 21:21:06.391133 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.391107    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:21:06.395598 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.395574    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-590ada-84b655d88-xv9hn"]
Jul 28 21:21:06.416542 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.416507    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:21:06.428589 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:06.428549    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-590ada-694bb9b-5587r"]
Jul 28 21:21:07.301905 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:07.301859    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bba1b34-7d74-4b5e-97dd-58893340d479" path="/var/lib/kubelet/pods/9bba1b34-7d74-4b5e-97dd-58893340d479/volumes"
Jul 28 21:21:07.302351 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:07.302329    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a607b0c7-d737-4b14-8c70-3b476b96f16e" path="/var/lib/kubelet/pods/a607b0c7-d737-4b14-8c70-3b476b96f16e/volumes"
Jul 28 21:21:12.529531 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:12.529489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:12.635769 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:12.635735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:12.857496 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:12.857397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:13.072713 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:13.072678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:14.010536 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:14.010499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:24.079352 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.079312    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:21:24.079773 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.079541    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" podUID="799608e8-cacf-4c35-aedf-1d7ff2f33853" containerName="maas-api" containerID="cri-o://502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51" gracePeriod=30
Jul 28 21:21:24.131319 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.131284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:24.140962 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.140930    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"]
Jul 28 21:21:24.153184 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.153155    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-96b3c5-29754555-z54vk"]
Jul 28 21:21:24.223421 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.223386    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:21:24.223649 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.223623    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" podUID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" containerName="payload-processing" containerID="cri-o://003842b4c89d12db4d8787d2bfbe601e0e6817bd9c6cc8b51033cc204222ff08" gracePeriod=30
Jul 28 21:21:24.307503 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.307462    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:24.367715 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.367689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:21:24.467552 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.467499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:24.472082 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.472056    2576 generic.go:356] "Generic (PLEG): container finished" podID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" containerID="003842b4c89d12db4d8787d2bfbe601e0e6817bd9c6cc8b51033cc204222ff08" exitCode=0
Jul 28 21:21:24.472198 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.472125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" event={"ID":"971f72e8-ec1d-4d75-9f45-5a2ae28394ba","Type":"ContainerDied","Data":"003842b4c89d12db4d8787d2bfbe601e0e6817bd9c6cc8b51033cc204222ff08"}
Jul 28 21:21:24.473440 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.473419    2576 generic.go:356] "Generic (PLEG): container finished" podID="799608e8-cacf-4c35-aedf-1d7ff2f33853" containerID="502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51" exitCode=0
Jul 28 21:21:24.473526 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.473483    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"
Jul 28 21:21:24.473526 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.473499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" event={"ID":"799608e8-cacf-4c35-aedf-1d7ff2f33853","Type":"ContainerDied","Data":"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"}
Jul 28 21:21:24.473621 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.473546    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps" event={"ID":"799608e8-cacf-4c35-aedf-1d7ff2f33853","Type":"ContainerDied","Data":"95ee5477bdf91a1fa53a1819057f756f0d54ab52609e73cc13e9bcce3d68ff28"}
Jul 28 21:21:24.473621 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.473570    2576 scope.go:117] "RemoveContainer" containerID="502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"
Jul 28 21:21:24.487202 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.487177    2576 scope.go:117] "RemoveContainer" containerID="502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"
Jul 28 21:21:24.487509 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:21:24.487489    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51\": container with ID starting with 502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51 not found: ID does not exist" containerID="502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"
Jul 28 21:21:24.487594 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.487517    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51"} err="failed to get container status \"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51\": rpc error: code = NotFound desc = could not find container \"502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51\": container with ID starting with 502513baec07c1910f1e9f4f984fee568d980b9de6690934182f789898f84f51 not found: ID does not exist"
Jul 28 21:21:24.488691 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.488674    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:21:24.489009 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.488994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq8ks\" (UniqueName: \"kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks\") pod \"799608e8-cacf-4c35-aedf-1d7ff2f33853\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") "
Jul 28 21:21:24.489080 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.489039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls\") pod \"799608e8-cacf-4c35-aedf-1d7ff2f33853\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") "
Jul 28 21:21:24.489173 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.489154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca\") pod \"799608e8-cacf-4c35-aedf-1d7ff2f33853\" (UID: \"799608e8-cacf-4c35-aedf-1d7ff2f33853\") "
Jul 28 21:21:24.489577 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.489551    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "799608e8-cacf-4c35-aedf-1d7ff2f33853" (UID: "799608e8-cacf-4c35-aedf-1d7ff2f33853"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:21:24.490969 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.490948    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "799608e8-cacf-4c35-aedf-1d7ff2f33853" (UID: "799608e8-cacf-4c35-aedf-1d7ff2f33853"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:21:24.491069 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.491051    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks" (OuterVolumeSpecName: "kube-api-access-pq8ks") pod "799608e8-cacf-4c35-aedf-1d7ff2f33853" (UID: "799608e8-cacf-4c35-aedf-1d7ff2f33853"). InnerVolumeSpecName "kube-api-access-pq8ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:21:24.590850 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.590771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume\") pod \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") "
Jul 28 21:21:24.591006 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.590884    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfh8w\" (UniqueName: \"kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w\") pod \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\" (UID: \"971f72e8-ec1d-4d75-9f45-5a2ae28394ba\") "
Jul 28 21:21:24.591141 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.591119    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "971f72e8-ec1d-4d75-9f45-5a2ae28394ba" (UID: "971f72e8-ec1d-4d75-9f45-5a2ae28394ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:21:24.591180 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.591132    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq8ks\" (UniqueName: \"kubernetes.io/projected/799608e8-cacf-4c35-aedf-1d7ff2f33853-kube-api-access-pq8ks\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:24.591180 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.591167    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/799608e8-cacf-4c35-aedf-1d7ff2f33853-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:24.591180 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.591178    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/799608e8-cacf-4c35-aedf-1d7ff2f33853-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:24.592868 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.592850    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w" (OuterVolumeSpecName: "kube-api-access-bfh8w") pod "971f72e8-ec1d-4d75-9f45-5a2ae28394ba" (UID: "971f72e8-ec1d-4d75-9f45-5a2ae28394ba"). InnerVolumeSpecName "kube-api-access-bfh8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:21:24.601343 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.601316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:24.692616 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.692585    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfh8w\" (UniqueName: \"kubernetes.io/projected/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-kube-api-access-bfh8w\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:24.692616 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.692610    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/971f72e8-ec1d-4d75-9f45-5a2ae28394ba-plugins-config-volume\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:24.801312 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.801283    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:21:24.806017 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:24.805991    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-96b3c5-586b4b5b9c-lfzps"]
Jul 28 21:21:25.034085 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.034058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:25.301265 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.301180    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ced2ad-16a8-4c8d-be87-b82933b778d5" path="/var/lib/kubelet/pods/35ced2ad-16a8-4c8d-be87-b82933b778d5/volumes"
Jul 28 21:21:25.301628 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.301546    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799608e8-cacf-4c35-aedf-1d7ff2f33853" path="/var/lib/kubelet/pods/799608e8-cacf-4c35-aedf-1d7ff2f33853/volumes"
Jul 28 21:21:25.481083 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.481048    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"
Jul 28 21:21:25.481271 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.481048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh" event={"ID":"971f72e8-ec1d-4d75-9f45-5a2ae28394ba","Type":"ContainerDied","Data":"4ecfd58e58b8bae27833b5982561791d42a70ba583ae9ba05f77f9e71b479428"}
Jul 28 21:21:25.481271 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.481229    2576 scope.go:117] "RemoveContainer" containerID="003842b4c89d12db4d8787d2bfbe601e0e6817bd9c6cc8b51033cc204222ff08"
Jul 28 21:21:25.512743 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.512715    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:21:25.519719 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:25.519697    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-96b3c5-5d4b678dc4-h5dkh"]
Jul 28 21:21:27.301076 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:27.301040    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" path="/var/lib/kubelet/pods/971f72e8-ec1d-4d75-9f45-5a2ae28394ba/volumes"
Jul 28 21:21:28.876804 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:28.876766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:31.583117 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:31.583064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:31.714305 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:31.714266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:31.909502 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:31.909419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:32.175585 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:32.175489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:37.163570 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.163527    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:21:37.164113 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.163808    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" podUID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" containerName="maas-api" containerID="cri-o://8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79" gracePeriod=30
Jul 28 21:21:37.190977 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.190947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:37.293129 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.293096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:37.405622 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.405589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:37.419811 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.419752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:21:37.499276 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.499245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca\") pod \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") "
Jul 28 21:21:37.499509 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.499332    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls\") pod \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") "
Jul 28 21:21:37.499509 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.499395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz8ks\" (UniqueName: \"kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks\") pod \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\" (UID: \"8ba2ad54-b600-4fc8-8025-8a21be163cc1\") "
Jul 28 21:21:37.499701 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.499669    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ba2ad54-b600-4fc8-8025-8a21be163cc1" (UID: "8ba2ad54-b600-4fc8-8025-8a21be163cc1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 21:21:37.500141 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.500116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:37.501516 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.501488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks" (OuterVolumeSpecName: "kube-api-access-sz8ks") pod "8ba2ad54-b600-4fc8-8025-8a21be163cc1" (UID: "8ba2ad54-b600-4fc8-8025-8a21be163cc1"). InnerVolumeSpecName "kube-api-access-sz8ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 21:21:37.501627 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.501516    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ba2ad54-b600-4fc8-8025-8a21be163cc1" (UID: "8ba2ad54-b600-4fc8-8025-8a21be163cc1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 21:21:37.556444 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.556413    2576 generic.go:356] "Generic (PLEG): container finished" podID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" containerID="8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79" exitCode=0
Jul 28 21:21:37.556592 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.556459    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"
Jul 28 21:21:37.556592 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.556513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" event={"ID":"8ba2ad54-b600-4fc8-8025-8a21be163cc1","Type":"ContainerDied","Data":"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"}
Jul 28 21:21:37.556592 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.556566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896" event={"ID":"8ba2ad54-b600-4fc8-8025-8a21be163cc1","Type":"ContainerDied","Data":"bbd8be6bdc123a35777061cce2b84cc1c9073c55f44e32d55d7e80358544c785"}
Jul 28 21:21:37.556592 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.556588    2576 scope.go:117] "RemoveContainer" containerID="8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"
Jul 28 21:21:37.565840 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.565823    2576 scope.go:117] "RemoveContainer" containerID="8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"
Jul 28 21:21:37.566082 ip-10-0-139-131 kubenswrapper[2576]: E0728 21:21:37.566063    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79\": container with ID starting with 8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79 not found: ID does not exist" containerID="8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"
Jul 28 21:21:37.566150 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.566092    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79"} err="failed to get container status \"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79\": rpc error: code = NotFound desc = could not find container \"8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79\": container with ID starting with 8e6f7283dd7aa26bbf1e64650703e799f4aaee5fbe2fb93904e2a40b43c53a79 not found: ID does not exist"
Jul 28 21:21:37.586553 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.586527    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:21:37.595052 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.595030    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-4e7a7a-76ccd9c97-6r896"]
Jul 28 21:21:37.600844 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.600823    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba2ad54-b600-4fc8-8025-8a21be163cc1-maas-api-tls\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:37.600939 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.600847    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz8ks\" (UniqueName: \"kubernetes.io/projected/8ba2ad54-b600-4fc8-8025-8a21be163cc1-kube-api-access-sz8ks\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:37.600939 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.600861    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba2ad54-b600-4fc8-8025-8a21be163cc1-openshift-service-ca\") on node \"ip-10-0-139-131.ec2.internal\" DevicePath \"\""
Jul 28 21:21:37.933761 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:37.933726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:39.300881 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:39.300845    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" path="/var/lib/kubelet/pods/8ba2ad54-b600-4fc8-8025-8a21be163cc1/volumes"
Jul 28 21:21:45.029819 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:45.029778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:45.151834 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:45.151792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:45.353728 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:45.353644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:45.604149 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:45.604065    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:46.074789 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:46.074755    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9747db87b-rzcjv_5386f485-3a20-4f69-828d-73af48e84cb0/authorino/0.log"
Jul 28 21:21:48.793707 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:48.793670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:21:51.936241 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:51.936198    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4tcks_7748fbf0-5dc4-4586-8191-91e14af8ab11/manager/0.log"
Jul 28 21:21:52.068930 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.068891    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h2gvl_8cefa4ed-509f-4ee8-97c2-aa0b01b290fd/manager/0.log"
Jul 28 21:21:52.209080 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.208985    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57b5564d8b-wpfd5_cbc1b530-a7c3-4913-9aab-71c044ba51f0/manager/0.log"
Jul 28 21:21:52.347936 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.347911    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9996g_488a6ede-7923-4a5d-911c-11425d3e2ced/server/0.log"
Jul 28 21:21:52.475339 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.475253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-84b9p_bbdc44c6-c6ff-4661-8017-96def9de0514/manager/0.log"
Jul 28 21:21:52.789280 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.789257    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56794f8cc8-dgcqq_cad0a907-5f5c-4903-8987-d7b278f7c43d/copy-manifests/0.log"
Jul 28 21:21:52.800750 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:52.800728    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56794f8cc8-dgcqq_cad0a907-5f5c-4903-8987-d7b278f7c43d/manager/0.log"
Jul 28 21:21:53.373631 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:53.373602    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8b5db48f9-6wdg7_b08c9497-a4a3-4713-a485-8678fbc1359d/maas-api/0.log"
Jul 28 21:21:53.508927 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:53.508897    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754525-vcmbp_3fa26325-49e2-4a8e-9849-040b941ac1be/cleanup/0.log"
Jul 28 21:21:53.638249 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:53.638160    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754540-zr928_bccbeb55-89af-4d55-9941-34c6ce50b8e5/cleanup/0.log"
Jul 28 21:21:53.889127 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:53.889039    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xj7l5_9cf8f5a7-b377-4628-93f5-6cda90204416/postgres/0.log"
Jul 28 21:21:54.748990 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.748957    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/extract/0.log"
Jul 28 21:21:54.755830 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.755807    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/util/0.log"
Jul 28 21:21:54.762682 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.762663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/pull/0.log"
Jul 28 21:21:54.887463 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.887433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/util/0.log"
Jul 28 21:21:54.895543 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.895519    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/pull/0.log"
Jul 28 21:21:54.905015 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:54.904998    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/extract/0.log"
Jul 28 21:21:55.027344 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.027272    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/util/0.log"
Jul 28 21:21:55.036666 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.036632    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/pull/0.log"
Jul 28 21:21:55.044304 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.044282    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/extract/0.log"
Jul 28 21:21:55.184132 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.184101    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/util/0.log"
Jul 28 21:21:55.192076 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.192058    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/pull/0.log"
Jul 28 21:21:55.200196 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.200179    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/extract/0.log"
Jul 28 21:21:55.333871 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.333780    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9747db87b-rzcjv_5386f485-3a20-4f69-828d-73af48e84cb0/authorino/0.log"
Jul 28 21:21:55.459948 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.459910    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5v9gj_c3b12d86-225c-4e16-a9a2-c21431769eed/manager/0.log"
Jul 28 21:21:55.577329 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.577298    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-979gz_eb62b356-3ee3-429b-90b3-b7e7edefc339/manager/0.log"
Jul 28 21:21:55.709089 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.709011    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rgws2_3e757894-4a7a-4833-a51e-91cd5a99b98d/kuadrant-console-plugin/0.log"
Jul 28 21:21:55.857500 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:55.857468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hdshq_3ea85666-dc43-4f69-aef0-46777c0dd0bf/registry-server/0.log"
Jul 28 21:21:56.011801 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:56.011771    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7k5fb_16fffe17-9517-4d3e-b5f1-7deda336a128/manager/0.log"
Jul 28 21:21:56.271573 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:56.271491    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lfx84_7155e9c6-3e38-4a52-bf08-61a29e326819/manager/0.log"
Jul 28 21:21:56.898181 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:56.898150    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d6cc5795b-cxh4l_32ec4a8d-5e06-4818-b37f-0fcda891eb77/kube-auth-proxy/0.log"
Jul 28 21:21:57.896546 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:57.896510    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jwxb5_4181c467-5cda-49c1-bc02-a6279a0459ae/main/0.log"
Jul 28 21:21:58.274177 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:21:58.274149    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvcmv7_81466578-4350-4bb8-afa4-fb1004e28776/main/0.log"
Jul 28 21:22:05.123662 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.123622    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hqh9t/must-gather-bg8cn"]
Jul 28 21:22:05.124040 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124027    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2975c36a-2fcc-4797-b6fd-0738f45df246" containerName="payload-processing"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124042    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2975c36a-2fcc-4797-b6fd-0738f45df246" containerName="payload-processing"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124051    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d537f5-cf4b-458b-83e9-e68cd86e8940" containerName="maas-api"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124057    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d537f5-cf4b-458b-83e9-e68cd86e8940" containerName="maas-api"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124067    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799608e8-cacf-4c35-aedf-1d7ff2f33853" containerName="maas-api"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124073    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="799608e8-cacf-4c35-aedf-1d7ff2f33853" containerName="maas-api"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124088    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" containerName="payload-processing"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124093    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" containerName="payload-processing"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124100    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a607b0c7-d737-4b14-8c70-3b476b96f16e" containerName="maas-api"
Jul 28 21:22:05.124102 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124105    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a607b0c7-d737-4b14-8c70-3b476b96f16e" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124113    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124117    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124125    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae8f2970-6958-4825-a237-90166f7c523d" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124130    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae8f2970-6958-4825-a237-90166f7c523d" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124136    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bba1b34-7d74-4b5e-97dd-58893340d479" containerName="payload-processing"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124143    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bba1b34-7d74-4b5e-97dd-58893340d479" containerName="payload-processing"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124198    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba2ad54-b600-4fc8-8025-8a21be163cc1" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124205    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d537f5-cf4b-458b-83e9-e68cd86e8940" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124211    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="971f72e8-ec1d-4d75-9f45-5a2ae28394ba" containerName="payload-processing"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124219    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae8f2970-6958-4825-a237-90166f7c523d" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124224    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a607b0c7-d737-4b14-8c70-3b476b96f16e" containerName="maas-api"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124232    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2975c36a-2fcc-4797-b6fd-0738f45df246" containerName="payload-processing"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124237    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bba1b34-7d74-4b5e-97dd-58893340d479" containerName="payload-processing"
Jul 28 21:22:05.124388 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.124242    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="799608e8-cacf-4c35-aedf-1d7ff2f33853" containerName="maas-api"
Jul 28 21:22:05.126345 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.126326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.138056 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.138031    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqh9t\"/\"kube-root-ca.crt\""
Jul 28 21:22:05.138181 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.138103    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqh9t\"/\"openshift-service-ca.crt\""
Jul 28 21:22:05.160090 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.160061    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqh9t/must-gather-bg8cn"]
Jul 28 21:22:05.192708 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.192676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:22:05.252160 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.252123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rjzx\" (UniqueName: \"kubernetes.io/projected/c38e911e-a793-4f73-b759-eb9dd04d5c1f-kube-api-access-7rjzx\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.252336 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.252174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c38e911e-a793-4f73-b759-eb9dd04d5c1f-must-gather-output\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.344681 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.344648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log" limit=1024
Jul 28 21:22:05.352923 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.352902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c38e911e-a793-4f73-b759-eb9dd04d5c1f-must-gather-output\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.353052 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.352967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rjzx\" (UniqueName: \"kubernetes.io/projected/c38e911e-a793-4f73-b759-eb9dd04d5c1f-kube-api-access-7rjzx\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.353241 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.353224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c38e911e-a793-4f73-b759-eb9dd04d5c1f-must-gather-output\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.362396 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.362360    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqh9t\"/\"kube-root-ca.crt\""
Jul 28 21:22:05.372713 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.372695    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hqh9t\"/\"openshift-service-ca.crt\""
Jul 28 21:22:05.381831 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.381775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rjzx\" (UniqueName: \"kubernetes.io/projected/c38e911e-a793-4f73-b759-eb9dd04d5c1f-kube-api-access-7rjzx\") pod \"must-gather-bg8cn\" (UID: \"c38e911e-a793-4f73-b759-eb9dd04d5c1f\") " pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.435873 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.435841    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqh9t/must-gather-bg8cn"
Jul 28 21:22:05.613672 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.613605    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqh9t/must-gather-bg8cn"]
Jul 28 21:22:05.615985 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:22:05.615947    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc38e911e_a793_4f73_b759_eb9dd04d5c1f.slice/crio-b093a8b5acdae0f5850429f0e6cf985ef46886f6da2ffa2cedd388a99fe4588f WatchSource:0}: Error finding container b093a8b5acdae0f5850429f0e6cf985ef46886f6da2ffa2cedd388a99fe4588f: Status 404 returned error can't find the container with id b093a8b5acdae0f5850429f0e6cf985ef46886f6da2ffa2cedd388a99fe4588f
Jul 28 21:22:05.620793 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.618978    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 21:22:05.724667 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.724571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqh9t/must-gather-bg8cn" event={"ID":"c38e911e-a793-4f73-b759-eb9dd04d5c1f","Type":"ContainerStarted","Data":"b093a8b5acdae0f5850429f0e6cf985ef46886f6da2ffa2cedd388a99fe4588f"}
Jul 28 21:22:05.914725 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:05.914687    2576 scope.go:117] "RemoveContainer" containerID="eb4d1bb4d3b0b5551063a27489ede4e2e5f18c9b49540119395314da88201b4d"
Jul 28 21:22:06.733726 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:06.733625    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqh9t/must-gather-bg8cn" event={"ID":"c38e911e-a793-4f73-b759-eb9dd04d5c1f","Type":"ContainerStarted","Data":"35e1dbdfbad6f73855792eb38ef91eb0c7adbcb5c917ed720ae71cb6222bb807"}
Jul 28 21:22:06.733726 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:06.733686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqh9t/must-gather-bg8cn" event={"ID":"c38e911e-a793-4f73-b759-eb9dd04d5c1f","Type":"ContainerStarted","Data":"9ef3f133479b2e05a8955e814ba0701d641396a527d031a43e525a8919657fab"}
Jul 28 21:22:06.763752 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:06.763695    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hqh9t/must-gather-bg8cn" podStartSLOduration=0.993711184 podStartE2EDuration="1.763676964s" totalImagesPullingTime="769.96578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 21:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:22:06.761683762 +0000 UTC m=+3421.946788469" watchObservedRunningTime="2026-07-28 21:22:06.763676964 +0000 UTC m=+3421.948781712"
Jul 28 21:22:08.177282 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:08.177251    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6746s_3144fb9b-2e19-46b6-ad95-b25777f15fc2/global-pull-secret-syncer/0.log"
Jul 28 21:22:08.366743 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:08.366687    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2hz8x_9fc2c721-7268-47a3-90ea-465677616a67/konnectivity-agent/0.log"
Jul 28 21:22:08.549665 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:08.549632    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-131.ec2.internal_29f9052eaaed67906c3e76b084a6751a/haproxy/0.log"
Jul 28 21:22:12.462435 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.462397    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/extract/0.log"
Jul 28 21:22:12.501533 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.501498    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/util/0.log"
Jul 28 21:22:12.531581 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.531543    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cglzs_bfd11f4b-5436-40a3-9205-7dded322e886/pull/0.log"
Jul 28 21:22:12.574099 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.574064    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/extract/0.log"
Jul 28 21:22:12.601198 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.601165    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/util/0.log"
Jul 28 21:22:12.629793 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.629757    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tdq4p_67119a4f-33d7-427b-8fe6-19bf9af34a2a/pull/0.log"
Jul 28 21:22:12.664594 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.664536    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/extract/0.log"
Jul 28 21:22:12.696386 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.696254    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/util/0.log"
Jul 28 21:22:12.720719 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.720632    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ktkm9_87c79d7f-8295-45f3-a0cb-cb562cff5e93/pull/0.log"
Jul 28 21:22:12.755808 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.755773    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/extract/0.log"
Jul 28 21:22:12.791108 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.791071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/util/0.log"
Jul 28 21:22:12.828980 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:12.828951    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fszhc_0e331bf8-bff4-46e0-b25d-7a0a730ccdc3/pull/0.log"
Jul 28 21:22:13.305451 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.305185    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9747db87b-rzcjv_5386f485-3a20-4f69-828d-73af48e84cb0/authorino/0.log"
Jul 28 21:22:13.346734 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.346696    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5v9gj_c3b12d86-225c-4e16-a9a2-c21431769eed/manager/0.log"
Jul 28 21:22:13.376560 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.376518    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-979gz_eb62b356-3ee3-429b-90b3-b7e7edefc339/manager/0.log"
Jul 28 21:22:13.419071 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.419038    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rgws2_3e757894-4a7a-4833-a51e-91cd5a99b98d/kuadrant-console-plugin/0.log"
Jul 28 21:22:13.490757 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.490727    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hdshq_3ea85666-dc43-4f69-aef0-46777c0dd0bf/registry-server/0.log"
Jul 28 21:22:13.722877 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.722678    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7k5fb_16fffe17-9517-4d3e-b5f1-7deda336a128/manager/0.log"
Jul 28 21:22:13.846381 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:13.846342    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lfx84_7155e9c6-3e38-4a52-bf08-61a29e326819/manager/0.log"
Jul 28 21:22:15.768990 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:15.768957    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ssrt2_882e3cc3-edc1-4141-8b27-9f5b9d604a9e/kube-state-metrics/0.log"
Jul 28 21:22:15.793790 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:15.793740    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ssrt2_882e3cc3-edc1-4141-8b27-9f5b9d604a9e/kube-rbac-proxy-main/0.log"
Jul 28 21:22:15.817609 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:15.817575    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-ssrt2_882e3cc3-edc1-4141-8b27-9f5b9d604a9e/kube-rbac-proxy-self/0.log"
Jul 28 21:22:15.850676 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:15.850644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bdcf88c4d-jjq5x_e90ca0a3-0b1d-4f18-abc8-9e654c8afe61/metrics-server/0.log"
Jul 28 21:22:16.011335 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.011280    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cf9st_4a3bc548-7704-4a1b-829d-6910659fba19/node-exporter/0.log"
Jul 28 21:22:16.044460 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.044331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cf9st_4a3bc548-7704-4a1b-829d-6910659fba19/kube-rbac-proxy/0.log"
Jul 28 21:22:16.069358 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.069326    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cf9st_4a3bc548-7704-4a1b-829d-6910659fba19/init-textfile/0.log"
Jul 28 21:22:16.585865 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.585834    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f566ddcb6-mz56w_76bd763a-bb1d-4bfc-81f7-6553ae63b6f3/telemeter-client/0.log"
Jul 28 21:22:16.614021 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.613986    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f566ddcb6-mz56w_76bd763a-bb1d-4bfc-81f7-6553ae63b6f3/reload/0.log"
Jul 28 21:22:16.638553 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:16.638514    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f566ddcb6-mz56w_76bd763a-bb1d-4bfc-81f7-6553ae63b6f3/kube-rbac-proxy/0.log"
Jul 28 21:22:17.989925 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:17.989887    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"]
Jul 28 21:22:17.993785 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:17.993758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:17.996520 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:17.996494    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hqh9t\"/\"default-dockercfg-7vqqt\""
Jul 28 21:22:18.003165 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.003140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"]
Jul 28 21:22:18.095814 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.095769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-podres\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.096024 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.095854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znppx\" (UniqueName: \"kubernetes.io/projected/75243cf9-2209-431c-89ca-c07d9acf8985-kube-api-access-znppx\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.096024 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.095947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-proc\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.096024 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.095979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-lib-modules\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.096218 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.096064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-sys\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197079 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znppx\" (UniqueName: \"kubernetes.io/projected/75243cf9-2209-431c-89ca-c07d9acf8985-kube-api-access-znppx\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197385 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-proc\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197481 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-lib-modules\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197541 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-sys\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197541 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-proc\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197657 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-podres\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197657 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197595    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-sys\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197762 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-lib-modules\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.197762 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.197690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75243cf9-2209-431c-89ca-c07d9acf8985-podres\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.205988 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.205960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znppx\" (UniqueName: \"kubernetes.io/projected/75243cf9-2209-431c-89ca-c07d9acf8985-kube-api-access-znppx\") pod \"perf-node-gather-daemonset-xntx7\" (UID: \"75243cf9-2209-431c-89ca-c07d9acf8985\") " pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.309994 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.309907    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.514213 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.510004    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"]
Jul 28 21:22:18.524228 ip-10-0-139-131 kubenswrapper[2576]: W0728 21:22:18.524194    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75243cf9_2209_431c_89ca_c07d9acf8985.slice/crio-c66d26ebd2bd41afb798968c4d753722fa8088b923cdd5e905298de058cb5c27 WatchSource:0}: Error finding container c66d26ebd2bd41afb798968c4d753722fa8088b923cdd5e905298de058cb5c27: Status 404 returned error can't find the container with id c66d26ebd2bd41afb798968c4d753722fa8088b923cdd5e905298de058cb5c27
Jul 28 21:22:18.847171 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.847078    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:18.847171 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.847147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7" event={"ID":"75243cf9-2209-431c-89ca-c07d9acf8985","Type":"ContainerStarted","Data":"ba022e7c9cd589afb9fa885ddfe64e65b996a20e037581d58e8309693e6b667f"}
Jul 28 21:22:18.847460 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.847177    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7" event={"ID":"75243cf9-2209-431c-89ca-c07d9acf8985","Type":"ContainerStarted","Data":"c66d26ebd2bd41afb798968c4d753722fa8088b923cdd5e905298de058cb5c27"}
Jul 28 21:22:18.869602 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:18.869541    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7" podStartSLOduration=1.8695209849999999 podStartE2EDuration="1.869520985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 21:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 21:22:18.865500911 +0000 UTC m=+3434.050605629" watchObservedRunningTime="2026-07-28 21:22:18.869520985 +0000 UTC m=+3434.054625693"
Jul 28 21:22:19.340183 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:19.340151    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-587465f6b9-smp5n_a07cde6c-9eca-4f3b-9789-8b09e05bf3a0/console/0.log"
Jul 28 21:22:21.400046 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:21.400013    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqjsk_52d5564d-7c12-46fb-bf0c-9d285bd5f7d9/dns/0.log"
Jul 28 21:22:21.436038 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:21.436007    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vqjsk_52d5564d-7c12-46fb-bf0c-9d285bd5f7d9/kube-rbac-proxy/0.log"
Jul 28 21:22:21.654189 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:21.654106    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpvlz_3746f2e3-5279-4ff3-85af-b46d1718e6be/dns-node-resolver/0.log"
Jul 28 21:22:22.291665 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:22.291631    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6cdd7ddc4b-ccfjl_4cc0a2c6-643f-43d4-a19c-75def068ffdf/registry/0.log"
Jul 28 21:22:22.408221 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:22.408187    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mbvjh_b23d94ba-ba07-4fa3-9fa7-94ce25677eeb/node-ca/0.log"
Jul 28 21:22:23.695693 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:23.695659    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d6cc5795b-cxh4l_32ec4a8d-5e06-4818-b37f-0fcda891eb77/kube-auth-proxy/0.log"
Jul 28 21:22:24.871827 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:24.871795    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hqh9t/perf-node-gather-daemonset-xntx7"
Jul 28 21:22:24.960561 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:24.960530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8w26d_234d4cc1-a8ce-40f7-80e7-423f8706623a/serve-healthcheck-canary/0.log"
Jul 28 21:22:26.004769 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:26.004733    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5bbt_1ece9f1f-8051-4b3b-a516-60adc8b8df00/kube-rbac-proxy/0.log"
Jul 28 21:22:26.045593 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:26.045558    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5bbt_1ece9f1f-8051-4b3b-a516-60adc8b8df00/exporter/0.log"
Jul 28 21:22:26.084833 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:26.084797    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5bbt_1ece9f1f-8051-4b3b-a516-60adc8b8df00/extractor/0.log"
Jul 28 21:22:28.765055 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:28.765028    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4tcks_7748fbf0-5dc4-4586-8191-91e14af8ab11/manager/0.log"
Jul 28 21:22:29.038479 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.038376    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h2gvl_8cefa4ed-509f-4ee8-97c2-aa0b01b290fd/manager/0.log"
Jul 28 21:22:29.248426 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.248395    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d6cc5795b-cxh4l_32ec4a8d-5e06-4818-b37f-0fcda891eb77/kube-auth-proxy/0.log"
Jul 28 21:22:29.249273 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.249227    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-57b5564d8b-wpfd5_cbc1b530-a7c3-4913-9aab-71c044ba51f0/manager/0.log"
Jul 28 21:22:29.277590 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.277561    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9996g_488a6ede-7923-4a5d-911c-11425d3e2ced/server/0.log"
Jul 28 21:22:29.389079 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.388990    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-84b9p_bbdc44c6-c6ff-4661-8017-96def9de0514/manager/0.log"
Jul 28 21:22:29.480647 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.480619    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56794f8cc8-dgcqq_cad0a907-5f5c-4903-8987-d7b278f7c43d/manager/0.log"
Jul 28 21:22:29.508327 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:29.508294    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56794f8cc8-dgcqq_cad0a907-5f5c-4903-8987-d7b278f7c43d/copy-manifests/0.log"
Jul 28 21:22:30.952607 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:30.952512    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d6cc5795b-cxh4l_32ec4a8d-5e06-4818-b37f-0fcda891eb77/kube-auth-proxy/0.log"
Jul 28 21:22:31.014849 ip-10-0-139-131 kubenswrapper[2576]: I0728 21:22:31.014801    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nwkxc_a1fae4c6-ff2b-4186-b606-bd891863cdb9/openshift-lws-operator/0.log"